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ABSTRACT 

The report Iprovides an interim assessment and 
discussion of the activities of the Research for Better Schools' 
Employee-Based Career E^^ucation Program during FY The 
introduction (section 1) outlines the r^^port's format and summarizes 
the evaluation staff's activities* The summative evaluation overview 
(section 2) and the formative evalua^-ion overview (section 3) present 
the rationale for both types of evaluation and generally describe the 
processes used for each* Section U and ^, on student populations and 
instruments and procedures, describe the students in the activities 
of the career education program, the groups of students used for 
comparisons, and the tests ana means of administration used to make 
th4 comparison between the groups* Section 6 deals with pretest 
resyiilts for which the arialysis is completed. Section 7 deals 
specifically with the commo'n instruments being cooperatively 
developed by the four employer--based career education programs and 
the National Institute of-^Education* Section 8 presents summative 
evaluation results which deal with testing hypotheses relating to the 
career education program in general. Section 9 presents formative 
evaluation results which are specific -^o the Research for Better 
Schools* implementation of a career education program. Section 10 
summarizes the report and makes recommendations* (Author) 
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EXECUTIVE SUMMARY 

I. INTRODUCTION 

I 

This report represents a mid-year review of the evaluation process and 
results from the Research for Better Schools Career Education Program. Exten- 



sive da_t.a_^.kQlh,descriptivfi and -analyXie^ are presen-ted.- -Conclusioas are-d-rawn 
to the extent possible without end of the year posttest results. The report 
it.self is organized into ten sections which forTn four major topical groupings 
as follows: 

1. Evaluation Overview - Sections I. II. and III. 

2. Procedures - Sections IV. and V. 

3. Results - Sections VI. VII. XIII and IX. 

4. Discussion - Section X. 
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The evaluation plan for FY 1974 was built around the component structure 
of the project, wherein fifteen separate components were specified. The major 
component groupings were Management Systems, Support Systems and Instructional 
Systems. The third group was intended to be the focus of the evaluation effort, 

Several facets of evaluation have received emphasis over the course of 
the year. Briefly they are: ^ 

1. Cohiputorized processing of program records has been developed to a 
fairly sophisticated level to handle operational, evaluative and 
research tasks. 

2. Extensive effort has been put intd the development of instruments 
germane, but not confined, to the Career Education Program. 

A reporting system, which goes beyond the contractual requirements, 
has been instituted and is under revision with the goal of making 
evaluation results more useable. 
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II. SUMMATIVE EVALUATION REVIEW 

The suramative evaluation has been designed principally to determine the 
overnl 1 c-ffcr i s of the p rogram on studxjn-tir7~c-mpl:TTyeT^— in d Pcif Wtl?. Tt' alscr 
-addresses institutional feasibility and planning issues from a program-wide 
perspective. V 

Student effects hypotheses have been stipulated regarding development in 
basic skills, career maturity, and career knowledge* Other effects hypotheses 
relate ^to the sufficiency of employer resources, employer^attitude , parent 
attitude) institutional structures, program costs and program marketability* 
These hypotheses will be tested via. comparisons between experimental and con- 
trol groups on a series of formalized measures, and the use of informal instru- 
mentation, observation and unobtrusive measures* 

III. FORMATIVE ^VALUATION OVERVIEW 
The formati\Ap evaluation has been designed to gather information which 

is useful for program development knd project management. Formative results 
often form the basi^ for summative inquiries. Only the components directly 
related to the provision of instruction hava^been included in the formative 
evaluation design. They are: Employer Support, Employer Utilization, Basic 
Skills, Career Development, Career Guidance and Instructional Systems* Each 
of these components is to be evaluated with regard to rationale and conduct, 
objective ef f ect ivenesa, and cost. Results are tp be reported in individual 
task documents for each component over the courg^e of the year* 

IV. STUDENT POPULATIONS 

Four different student groups have been ^selected for analysis of the 

/ 

Career Education program* The first two are "experimental'' groups, while 
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the last! two are "control" groups: 

1. ACE Group - These students (76) were twelfth graders spending 

their se cond year in the~program. They ex p erience a compre-- — ~ - 

hensive educational program at the Academy for Career Education. 

2. ACE-Olney Group - These students (76) were tenth and eleventh 
graders spending their first year in the program. \^They have 
the core program (Career Development, Career Guidance, Basic 

V Skills) and receive .other courses and activities at their home 
^ school. > 

3. Comparison Group - These students (28) were eleventh graders 
who applied and were accepted to the program, but decided not 
to enroll. They participate in their home school program. 

4. Context Group - These students (81) were tenth, eleventh and 
twelfth graders who were randomly selected for comparative 
purposes. They participate in their home school program. 

All student groups, except the ACE Group,^ were drawn entirely from a 
large secondary school (Olney High School) in the Philadelphia School District. 

V. . INSTRUMENTS AND PROCEDURES 

A pretest-pQsttest instrument package was constructed for measurement 
of all students at the beginning and end of the school year. This consisted 
of the Comprehensive Tests of Basic Skills (CTBS), the Career Maturity Inven-- 
tory (CMI) , the Assessment of Student Attitudes Questionnaire (ASA), and the 
Student Demographic Data QuestionnaAre (SDQ) . The first two are atandardized , 
commercially available instruments. The last two are measures developed for 
this project* f 

V 

Another, series of instruments has been developed for the experimental 



students only^ This consiAs of the Employer Cluster Tests, t;he Student 
Opinion Survey, the Parent Opinion Survey, the Employer Opinion Survey, the 
Emp 1 oy e r Qi^^^i s t and_Interview, the Career Exploration Student Questionnaire, 
and a number of forms. These instruments are administered in varying cycles.^ 
Their principal use is in formative evaluation although synthesized results 
have sununative implications. 

The pretest-posttest instruments are administered by the evaluation staff 
under standardized conditions* Other instruments are administered by the 
evaluation staff and operat»iona«l staff members. , 

VI. PRETEST RESULTS 

The Student Demographic Data Quest ionnaire *(SDQ) provided information 
regarding the background characteristics of Academy students and their counter- 
parts in the control groups. The students selected *seem to be representative 
of an urban, population. There appeared to be no between-group differences for 
previous school attendance or for parental occupation. Between-group' differ- 
ences were found for previous grade point averages (CPA) of the groups, post- 
secondary plans of the groups, and , the racial and sexual compositions of the 
groups. The ACE students had a CPA of C-, the ACE-(5d.ney group had a CPA of C, 
and the control groups averaged between the two experimental groups. All four 
groups showed a high level of interest in post-secondary education. Sex and 
racial data were collected only f or tj^^ ^^cademy groups. The control grpup 

students were more interested in immediate post-secondary employment than 

i 

were the experimental groups. The experimental groups also showed a greater 
diversity in the post-secondary planning than did the control groups. The ACE 
Group was approximately evenly divided between Males and Females; 70% of the 
ACE Group was Black and 30% was White. The ACE-Olney Group was 55% Male and 



45Z Fem^e; 80% of this group was Black and 20% was White* The Academy pro- 
gram secerns to be attracting significantly more Blacks than Whites* If this 
—tren d c ontinues^ the Academy program -will have served members of only one popu- 
" lat ion' group . The uneven "racial" compos inofT^ho'ulT^be s tuJieT~f ur ther • 

The Career Maturity Inventory (CMI) .was administered to all experimental 
and control groups. Three subtests of the CMI revealed distinct differences 
betwe'en the experimental and control grpiij)^* The subtests on which the ACE 
and ACE-Olney groups were equal to one another and significantly superior to 
the control groups were Occupational Information, Goal Selection, and Problem 
Solving* -The two other subtests of the CMI, Attitude S^ale and Planning, pre- 
sented inconclusive results with the ACE Group superior to the Comparison Group 
on one and the ACE and ACE-Olney groups superior to only the Context Group on 
the other* The significant differences on the three scales seem to be a factor 
in students* decisions to participate in the Academy program and warrant fur- 
ther investigation. 

The Comprehensive Tests of Basic Skills (CTBS) were also administered to 
all experimental and control groups* No significant differences between groups 
were demonstrated on the Arithmetic subtests* On each of the Reading subtests, 
the ACE Group was significantly different only from the Comparison Group; while 
it seems clear that the ACE students upon entrance to 11th grad!e were superior 
in reading skills to students who applied and dropped, impi^ications of this 
single comparison are difficult to draw* Other group comparisons support the 
contention that the experimental and control groups are equal in achievement 
on other basic skills* 

The Assessment of Student Attitudes Survey (ASA) was administered to 
all groups and revealed no differences between groups in their attitudes 
toward education, program curriculum, program counseling, or learning* The 
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only difference between groups was between the ACE and ACE-Olney groups in 
their attitude toward program resources; the ACE students appear to be more 



positive in their attitude toward program resources • This difference may be 
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attiributed-^o the grater length oh time the ACE Group has been enrolled in 
the experimental program 

The experimental and control groups were equal on many of the instruments 
administered. On the CTBS, all groups were statistically similar with the ex- 
ception that the ACE Group was superior to the Comparison Group on Reading sub- 
tests. The ASA results indicated that all groups were similar with the excep- 
tion that the ACE Group was superior to the ACE-Olney Group on the Attitude 
Toward Program Resources subscale. 

The CMI and background characteristics obtained from the SDQ di<J reveal 
some intergroup differences that warrant further study. The ACE and ACE-Olney 
groups were superior to both control groups on most subtests of the CMI, The 
V program also seems to be differentially attracting Black and White students, 
with Blacks forming a large majority of the students. Thus, both career ma- 
turity and race seem to be factors which differentiate between students who 
.display interest and enroll from those who display interest and do not enroll 
in the Academy program. Reasons for these differences necessitate further 
study. 

« 

VII. COMMON INSTRUMENT RESULTS 

Of the four common instruments designed for utilization at the EBCE 
sites, only the Student Opinion Survey had been administered in time for 
analysis and inclusion in the present report. Results from the other instru- 
. *ments will be reported in Special Reports, Task Reports and the Final Report. 
In general, students xated the Career Education Program highly with particular 
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emphasis on the progisJam^s interest /value , individualization and opportunity 
for learning. The more structured nature of- the RBS ptogram was /eflected in 



-stu d e nt s -lowe^^at i ngs of - abil i^y--t^ deter min e^— tAe^ mOun t - 
activities. The fact that these ratings were not extremely l^w, and that 
other perceptions were very favorable, would suggest that ^his is not a 
serious concern. It may not even be a criticism, but raxher a simple state- 
ment of fact. It should be pursued to conclusion just: in case a problem is 
there. Students evidenced a high vocational attitude, especially regarding 
their anticipation of employment. Most students /seemed to think that gaining 
knowledge about careers and jobs was the most important reason for entering 
the Career Education Program. Students were/ strongly favorable toward Ch^e 
Career Education Program in comparison witm traditional^school programs. When 
presented with a series of learning objectives, students thought all of. them 
were important. Interpersonal and social skills and self-evaluation were con- 
sidered the least important. The /atings of program success in achieving the 
ob jec^l^^S^were not as high as ^e rated importance of the objectives. The 
difference was fairly consistent, but difficult to interpret because the Zero 
Point for importance and ef f c^ctiveness may not be the same. For each objeqtive 
the ACE Group ratings of effectiveness were lower than the ratings accorded by 
the ACE-Olney Group. This was consistent with the other results obtained. 
Thus, it would seem that/ twelfth graders had a generally less favorable atti- 
tude toward the program than tenth and eleventh graders. Most students planned 
to get a high school/diploma. Only a minority were working during the school 
term, but some Group students wer.e working a substantial i^umber of hours 



per week. Ve&y few students replied that their vocational activities inter- 
fered witl)/any of their other activities. 



/ 
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Viri. INTERIM SUMMATIVE EVALUATION RESULTS 

This section has presented sTJramative results on the Kanageinent Systems 
Cumpo i ieiiLfc; ,— Support Sys^tems-Gomponents and pretest-postt^st data- oo- tw e l fth - — 
graders. Suinmative data at this time in the experimental year must be con- 
sidered tentative » Extensive analyses were not possible given the limited 
data available. Discussion of Management and Support systems was based largely 
on separate reports previously submitted for those areas. Pretest-post test 
results represented preliminary data on twelfth graders only. In Management 
Systems the following posit:^ve trends were noted: 

1. continuing' development of strong and useful Academy Board of 

Directors 

2. continuing licensure of the Academy ^ 

3. extensive ^ievelopment of relationship with the Philadelphia School 
* District 

4 

A, less centralization of project management 

5. development of a cost tracking system and approaches to cost analysis 

6. improvement in reporting system to NIE 

7, establishment of student comparison groups 

8, development of new information system ^ 
The following problem areas were also discussed: 

1, extent of policy codification necessary 

2, management of relations with the Chamber of Commerce 

3, development*, of a reporting system focused on evaluation usability 
Each of the areas listed in the operating plans as priorities was discussed. 

In Support S»ystems the following* positive developments were noted: 

1. maintenance of-adequate staffing for Vorkscope 

2. provision of better resources for articulation across teams and 
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components 

4. 

3. recruitment of a large and diverse student body for this year 

— m^rfrrtneTTaiice— of- ef fici en trl-og l s tic alr^y steins — 

5. continuation of effective Supplementary Program 
The following problem areas were also encountered: 

1. articulation between program staff and Chamber of 'Commerce stai£ 

2. needed improvement in training efforts 

More information on these issues may be found in Summative Reports 1 and 2. 

Analyses of the pretest-posttest results of the Comprehensive Tests of 
Basic Skills for twelfth grade students showed significant gains in most areas 
of reading and arithmetic'. The average gain was .6 of a year over a 5 month 
instructional period. Students who were not involved in Individualized Learn- 
ing Center activities did not farewell on these tests; in fact, an average 
loss was observed. The implications of these findings require fui^ther study. ' 
These results must be regarded as tentative since only a subgroup of students 
were tested and no control group data are available. 

IX. INTERIM FORMATIVE EVALUATION RESULTS 

The formative evaluation results of the report describe' the Career 
Education Program and presents formative evaluation results regarding the re- 
cruitment and selection of employers and site analysis, the support systems 
of the program, the use of adviso rs» the ras xui-tment and selection of students, 
the, instructional services provided to students, and the cost of providing in- 
structional services to students. 

The Research for Better Schools Career Education Program consists of four 
elements: the Career Development Uai t , the Career Guidance Unit, the Basic 
Skills Unit, and Supplementary Activities. The first three are available to 



all students in the program; the last is limited to 12th grade students, only, 
since Olney High School provides those aspects of the program to 10th and 11th 
grade students. 

These activities are provided or supported by 32 full-time professional 
staff. The staff fall into four categories: Program Administration (3), 
Evaluation (4) , Design and Development (13) , and Operations (12) . Staff were 
viewed as being capable in delivering all components of the program. The 
only staff area in need bf-^urther development concerned the complex relation- 
ships between RBS and the Chamber of Commerce • The tasks to be accomplished 
are demanding, and working relationships need clarified and strengthened 

^o that maximum efficiency may b6 achieved. Also relating to staff pre- 
operational training efforts were minimal due to the late date of finalization 
of the contract with the National Institute of Education. ' 

Facilities used in the Career Education ,Pro|ram embrace four major sites: 
The Academy for Career Education, the RBS offices, the Greater Philadelphia 
Chamber of, Commerce, and employer sites. The facilities are adec^uate for the 
needs of the current enrollment. 

The recruittont and selection o-f employers is based on the clustering 

J- 

of employers into areas of related activities. Last year there were 12 clusters 
This year A clusters have been added to the program, representing a 33 per cent 
increase in this type of activity. A total of 8A employers have been recruited 
and selected for participation in the Career Education Program. Of these, 53 
were recruited last year and 31 were recruited this year. Of the 53 partici- 
pating in FY 1973, lA did not participate in FY 197A. Reasons for non-parti- 
cipation include: the activities were not adequate, funds or staff were unavail 
able, students were perceived as insufficiently motivated, and the employers In- 
volved were in the process of an internal reorganization. An additional 9 em- 
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ployers have withdravm frojn the program in FY 1974. Reasons for these with- 
drawals include: the .activities provided were not adequate, funds or staff 
were unavailable, and involvement by the organization in a major building 
campaign. Of the 70 employers recruited and selected for participation in 
the FY 1974 Career Education Program, 61 or 87 per cent remain actively in- 
volved and interested in the program. 

Policy making Tor the Career Education Program is vested in the Board 
of Directors of the Academy for Career Education. The composition of this 
board includes representatives from industry, labor, education and the com- 
munity. Most advisory groups have been incorporated into the '^decision making 
process by granting representation on the Board of Dire€-to« of the Academy. 
In addition to traditional advisory groups already having membership on the 
Board, the offer of Board membership has been extended to parent and student 
groups. Other advisory groups include NIE, the other Experience-Based Career 
Education Programs, and the Board of Directors of Research for Better Schools. 

Student recruitment and selection in FY 1974 consisted of the selection 
and enrollment of lOth and 11th grade students, who would participate in the 
cor^^^aspects of the Career Education Program (Career Development, Career Guid- 
ance, and Ba'sic Skills) and receive the supplementary aspects of their program 
from their sending school. A total of 69 students were accepted and enrolled 
In-t^e Academy for Career Education program. An additional 92 students were 
accepted but declined to participate in the program. A total of 200 students 
were considered ineligible far the program. The greater portion of students 
accepted were Hth grade students. Acceptance was about equal by sex although 
more males than females enrolled. Few Whites applied for the program, and as 
a result 80 per cent of the enrollees were Black. If this racial trend con- 
tinues, the Career Education Program will have served members of only one 
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population subgroup.^ Analysis of data on previous grade average indicates 
that students with 70-89 averages who were accepted for the program decide 
to enroll, while students who were accepted with 60-69 averages declined to 
enroll. Two factors of the student recruitment process which warrant further 
investigation are the race factor and the apparent self-monitoring of decisions 
to" enroll based on previous grade averages. 

. A review of the instructional services provided during the fifth quarter 
shows that a total of 20,080 hours of instruction were scheduled and that 
16,688 hours of instruction were attended. Rates of attendance varied sharply 
within the units of the Career Education Program. Rates of attendance were 
reasonable for the Career Development Unit, the Career Guidance Unit, and the 
Supplementary Activities and well below expectation for the Basic Skills Unit. 
Students enrolled in activities which represent a total of 221.57 credits and 
actually earned 199.47 of those credits. Rates of earning credits were above 
80 per cent for all but 12th grade students in the Basic Skills Unit, where 
the rate was 71.5 per.^j:ent. The rate of earning credits is reflected in the 
grade averages of students in the various activities. Grade averages were at 
,y C+ level or above fpr 10th and 11th grade students for all activities. 
Twelfth grade students had a B^ or B- average in all activities except the Basic 
Skills Unit, where they had a C average. Areas of particular concern are the 
rate of earning pf credit by 12th grade students in the Basic Skills Unit and 
the rate of attendance by all students in the. Basic Skills Unit. 

The cost of providing instructional services was $602.07 per student; this 
is a weighted cost which accounts for 12th grade participation in the Supple- 
mentary Activities. The cost per student for the core program of Career Devel- 
opment, Career Guidance, and Basic Skills was $511.88. Botl^of these figures 
are per quarter. The cost per credit earned was $401.76. Total operational 
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expenditures in the Carc^er Education Program in the fifth quarter were' $91,148 
this figure includes an advance of $50,000 on a subcontract for instructional 
services. The average quarterly expenditure for the fifth and sixth quarters 
was $80,120.50. Projected costs for providing 12 months of instructional ser- 
vices to 10th and 11th grade students are $1;852.52 per stu-dent. Pg^^ected 
costs for proyidging 9 months of instructional services to 12th grade Student? 
graduating in June 1974 are $2,223.00 per student. Both these figures are 
below the anticipated yearly cost per student of $3,309.36. 



X. DISCUSSION AND RECOMMENDATIONS , ? ^ * f 



Selected program aspects have been discussed, and recommendations derived 
from consideration of the issues 'Involved. These recommendations wjere as 
follows: ; • ' / ' , . 

Re commend at ion 1 > Prior to funding next year, NIE should determine the 
content and extensiveness of major evaluation reports which are to be used for 
management and planning purposes.. This would better enable the incorporation 
of NIE^s interests into evaluation planning, and would result in more focused 
and usable evaluation reports. Such planning would have been difficult in the 
past, but, as the programs have achieved more resolution, such action has 
become more feasible. 

Recommendation 2 . After the nature and extent of reporting have been 
determined, a realistic level of resources should be allocated in the budget 
for the effort required. ^ , 

Recommendation 3. Every effort should be made to enable early re- 
cruiting of students, and selection should be made by random assignment of 
qualified candidates to experimental and control groups. 
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Recommendation 4> The status of existing instrumentation should be 
assessed. The characteristics of standardizedV instrumentation as they relate 
to' program objectives and the valid determination of comparative program 
effects should be investigated. A coordinated effort to maximize the 
technical soundness and generalizability of project-developed instrumentation 
. should be undertaken. ^ / 

Recommendation 5 . Project; staff should make a special effort through 
counseling, instructional and personal interaction activities to determine the 
nature and extent of dissatisfaction among twelfth graders. If the phenomenon 
can be causally defined, a concerted effort should be made to correct it. 

i 

Recommendation 6 . Since it is poss^ible that the observed phenomenon is 
an experimental effect, e.g., Hawthorne depreciation or ciimmulative measure- 
ment fatigue, it should be carefully observed next year, when comjiarable longity- 
dinal data may be collected.^ I 

Recommendation 7 . The evaluation task load should be reexamined and 
prioritized to definitely permit the production of user-referenced reports and 
assessment of evaluation utilization. The possibility of excizing some facets 
of the evaluation in order to enhance these priority areas should be considered ' 
' particularly for next year, , 

Recommendation 8. Possible program deficiencies in the areas of least 
evident basic skills student gains should be investigated. The program 
materials or its utilization may be manipulated to better serve apparent student 
needs . 

Recommendation 9. In view of the fact that students not in the Indivi- 
dualized Learning CentA^ seem to regress (some of it is artifactual) , non- 
assignment to this activity should be made very carefully. The alternatives to 
Center assignment should also be well considered. 



ERIC 

hmimiirnrrTiaaiia 



17 



XV 



Recommendation 10, Continuing efforts should be made to identify ex- 
perience clusters relevant to the Career Education Program and to recruit and 
select employers to both maintain and extend the cluster system. 

Recommendation 11, A total of 670 students should be identified who 
could be accepted for the Career Education Program; these 670 would meet the 
Career Education Program's needs for enrollment and control group students. 
The figure of 670 students was derived in the following manner: 
^ 520 = 2 X 1.3 X desired new enrollment of 200 
+ 150 = 1.5 desired control groups of 100 

670 = the number of students who must be identified as being acceptable 
for the program. 

Recommendation 12 . Since the selection of students for the Career 
Education Program is to be on a random basis next year, efforts to obtain a 
representative racial composition will have to be focused in the area of 
recruitment. Efforts to recruit students should be directed toward all 
racial subgroups of the population. 

Recommendation 13 . Efforts should be made to increase the motivation 
of students to both attend and achieve in Basic Skills Activities. Efforts 
to increase motivation have relied on intrinsic factors in the past; an in- 
vestigation of the possible application of a combination of intrinsic and 
extrinsic motivational factors to the Basic Skills Unit should be conducted. 

Recommendation 14 . Participation in the , specialization aspects of the 
Career Development and Career Guidance Programs should be strongly encouraged 
by the Counselor-Coordinators. Students should be made more aware of the 
personal benefits that can accrue to them by such participation. 

Recommendation 15. ' A more concisely defined relationship should be 
developed with the Greater Philadelphia Chamber of Commerce to ameliorate 
any cfonfusions regarding responsibility or accou'ntability for the development 



xvi 



and implementation of Career Development and Career Guidance activities. 
The roles of the team leaders of the Career Development and Career Guidance 
Units and the Project Director of the Chamber should be defined in terms „of 
explicit function and related responsibilities. 

Recommendation 16. Means of incorporating aspects of the .Supplementary 
Activities into the other units of the Career Education Program should be 
investigated. This recommendation might relate to means of increasing student 
motivation in the Basic Skills Unit (Recommendation 13) • 
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Summative Evaluation Report 
3 

INTERIM EVALUATION REPORT ^ 
I. INTRODUCTION 

The Interim Evaluation Report provides an interim assessment and discus- 
sion of the activities of the Research for Better Schools' Career Education 
Program during FY 1974. The Introduction (Section I) outlines the format and 
context of the report and capsulizes the activities of the evaluation staff 
over the past year. The Summative Evaluation Overview (Sectioti II) and the 
Formative Evaluation Overview (Section III) present the rationale for both 
types of evaluation and generally describe the processes used for each. The 
sections on Student Populations (Section IV) and Instruments an4 Procedures 
(Section V) describe the students in the activities of the Career Education 
Program, the groups of students used for comparisons, and the tests and means ^ 
of administration that will be used to make the comparisons between the groups 
Interim results are presented in Sections VI-IX of this report. Section VI 
deals with pretest results for which the analysis is completed. Sectioi^ij.'VII 
deals specifically with the common instruments which are being cooperatively 
developed by the four Experience-Based Career Education (EBCE) Programs and 
the National Institute of Education. Section VIII presents summative evalua- 
tion results which deal with testing of hypotheses which relate to the Career 

Education Program in general. And Section IX presents formative evaluatiog^ 

t 

results which are specific to the Research for Better Schools* implementation 
of a Career Education Program. Section X discusses and summarizes the infor- 
mation presented in other sections of the report. 



In its evaluation plan (append ed to Operating Plans for FY 1974 ) the 
Research for Better Schools Career Education Program was discussed within the 
framework of fifteen separate components. These components were linked into 
three groupings which were planned for as/ units. The first group was tenned 
VManagement Systems** and was composed of, the following components: 

1. Policy Determination 

2. Institutional Relationships 

3. Community Relations 

4. Planning and Formative Evaluation 

5. Program Administration and Management 

These components deal primarily with management and public relations. Tfte 
second group was termed **Support ^Systems" and was composed of the following 
components: 

1. Students 

2. Staff ' 

3. Logistics 

4. Supplementary programs ' 

These components'^ deal primarily with project inputs. The third group consistc 
o£ those components most central to the learning process intended by the 
program. It was composed of the following components: 

1. Employer Suppojrt 

2. Employer Utilization 

3. Basic Skills 

4. Career pevelopment Skills 

5. Guidance 

6. Instructional Systems 
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The analysis of this third group was intended to be the focus of this year's 
evaluation effort » The final product of the evaluation of each component 
would be an evaluation report on each component which would also be an input 
for the development of replication specif ications » It was intended that evalua- 
tion results would be fedback to staff in a usable form on a regular basis. 

The plan of evaluating the components identified above ]ias been followed 
in a manner similar to that prescribed in the "Evaluation Plan for FY 1974." 
In the process certain elements have become focal and taken on priority 
status* Although they do not fully describe the FY 1974 evaluation effort, 
a discussion of these emphasized elements may serve to characterize the ev^ua- 

lo- 
tion of RBS' Career Education Program. ^ V , 

Computerized processing of program records has been accorded much atten- 
tion and time/ All permanent records and much interim data on students have 
been prepared for machine processing. The. data system, described in outline 
in the Data Format Manual for FY 1974 , has become fully operational during the 
current project year. Several time-consuming manual processes are being phased 
out due to this automated capacity for. handling credits, grades, attendance, 
scheduling, test and questionnaire scoring and analysis, etc. The software 
for a sophisticated system of monitoring student absences and recording achieve- 
ment per unit time is presently being developed as an extention of the data 
system. The focus on automated data processing is considered important as a 
tool for managing the eventually larger student groups and enabling rigorous 

-i 

and .serious research on present and future program effects. 

Closely allied with the computer system has been the development of a 
unified forms system to assure accurate and appropriate input of records. The 
evaluation staff has undertaken the responsibility of developing, implementing, 

ERIC - ^ ?:2 



processing and coordinating all forms and records utilized within the program. 

This. has been done in an effort to reduce overlap, information gaps, and con- 

flicting procedures. A complete discussion of the design of this system may 

be found in Task Report 15A1 , ^ Develop Instructional Systems for Facilitating 

♦ 

the Integration, Utilization and Comn^unication of Learning Resources . 

Extensive resources have been applied ta instrument development. The 
following are principal among the instruments in ppocess 

1» Assessment of Student Attitudes Survey^ -/y 'designed as a pre-and pest- 
test of overall student attitude toward learning ^nd specific elements 
of the learning environment. ! 

2. Employer Cluster Tests ~ designed as pre-and posttests of student 
learning at employel^ sites each quarter. 

t 

3. Student Opinion Survey - designed as a measure of attitudes and, 

/' 

opinions, to be administered as a common instrument at each 
participating laboratory. 

4. ' Parent Opinion Survey - designed as a measure of attitudes and opinions 

to be administered as a common instrument at each participating lab- 
oratory. 

- 5. Employer Checklist and Interview - designed to monitor the employer 
resource recruitment, orientation and management process, as well as 
gather employer perceptions concerning the program. 
6. Student Needg and Interests Survey - designed to aid in student schedul 
ing and to monitor individualization. . 
The lack of appropriate instrumentation was one of the most formidable evalua- 
tion problems last year. Much has been done to remedy the situation this year, 
but the instrument development process will require at least another year for 
completion. 'J ' 



^ ' ^^^^ 

A flexible reporting system has been instituted in an att;empt to meet 
the needs of both program staff and the National Institupe of Education (NIE) . 
Summative and Formative Reporting Schedules will be described below. A 
third reporting vehicle, Special Reports, has been included to handle reports 
not in the original schedule but seen as necessary by the evaluation, pro- 
gram or NIE staff. The Special Reports thus far scheduled are: 

1. Report on Recruitment and Selection - 1/30/74 

2. Report on Student Characteristics - 2/28/74-'^ 

3. Revised Analysis Plan - 3/30/74 

4. Report on Instructional Units for Fifth Quarter - 3/30/74 
Others will be sch^uled as needs are made evident. 

These elements are incorporated into the evaluation plan, which attempts 
to monitor major processes and assess principal effects of the RBS Career 
Eduction Program. For a complete discussion of evaluation issues and projec- 
tions see Appendix A and Appendix B of the Operating Plans for FY 1974 . The 
sections below present more detail on summative and formative plans. 

II. SUMMATIVE EVALUATION OVERVIEW 
Of the many available definitional differences between summative and 
formative evaluation, none seems to be both clearly understandable and tech- 
nically precise. Therefore, a distinction v;ill be made here based upon 
differences that expeir^^ence suggests. These are practical lines of demarcation 
and any resemblance to theoretical issues is purely coincidental. For the 
purpose of this report, any evaluation activity which is related to a specific 
unit or subunit within the program will be consl^^l^red formative. Any evalua- 
tion activity which concerns the project as a whole or several units in a 



'sununary fashion will be .considered suinmative. For example, a report on the 
Career Development Unit employing orij*inal analyses of data related to that 
unit and confined to that unit would be formative • Anotlier report which 
reviewed previous analyses and documents related ta^^e Career Development 
Unit and discussed them in the context of other program units woi^ld be 
summative. ^ny evaluation activity involving /the study of effects which involve 
the program as a whole would also be summative. Formative evaluation intenda 
to inform the project staff of unit str)E^ngths and weaknesses, Formativg 
evaluation also suggests hypothesesyto be tested and problem areas to be 
assessed in summative evaluation^ Summative evaluation intends to judge 
project conduct and ef fective;?ieness , and to present results for external review* 

In its design the sunrihative evaluation for FY 1974 included a suiranary 
review of the Support ^2ystems and Management Systems Components. More attention 
than originally planed has been given to these components due to the' deletion 
from the work^cope of developmental reports on them; in many cases the evaluation 
report will be th^^nly documentation of these components. The summative focus, 
howler, will be on analyzing student effects. It is assumed that such .effects 
will be a result of the instructional components or the program as a whole; 
Instructional components will not be treated individually because there are 
no grounds for hypothesizing mutually exclusive effects* Analyst of costs and 
marketability have also been indicated in the summative plan. Both of these 
areas are embryonic and the evaluation is dependent upon administrative deci- 
sions regarding how cost and marketability are to be treated. In the least, 
a documentation and analysis of what occurred in these areas can be performed. 
Gathering and analyzing employer and parent data are also summative concerns* 
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A final area of Inquiry is the institutional structure necqs^ary to conduct 
a program of this type. 

Relating to the a^ove outline of summative concerns the following prin- 
cipal hypotheses are presented for testing during FY 1974: , 



Student Effects " 

' s 

' 1. Students will gain significantly (p<C.10) in basic skills over the 
course of the year. 
2/ Students will gain significantly (p<^.10) more in basic skills 
than comparable students in a traditional school. 

3. Students will gain significantly (p<^.10) in career maturity. 

4. Students will gain significantly (p<.10) more in career maturity 
than comparable students in a traditional school. 

5. Students will evidence a significantly (p<^.10> more positive attitude 
toward school than students in *a traditional school. 

I ,6. Students will gain significantly (p<^.10> in career knowledge over 
the course of cluster experiences. 

« 

other Eftect^^. ' V « .V 

1. Employers will be able to provide learning experiences sTuffici^t to 
meet student needs and interests . ' * 

2. Employers will evidence a positive attitude and coiranitrnent regarding 
the program. **^ti 

3. Parents will evidence a positive attitude and commitragint regarding 
the program. . ■ 
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4. Institutional structures will be established to enable the conduct 
of the program* 

5. It Vtll be demonstrated that the program can be operated on a feasible 
cost basis* 

6* It will be demonstrated that there is a ready market for the program. 
The student effects hypptheses will be tested using the experimental and 
control groups with instruments as discussed below in procedural sections. 
The design is a small scale, qxiasi-^-experimental one. The other hypotheses are 

not amenable to traditional statistical testing, but all available data will 

V 

be presented to argue a conclusion. Results will be discussed in the follow- 
ing summative evaluation reports: 

• REPORT , DUE DATE 

1^„ Report on Management Systems Components 1/ 2/74 

2. Report on Support Systems Components " 2/28/74 

3. Interim Evaluation Report 3/15/74 

4. Book of Measures 7/ 1/74 

5. Report on Instructional Systems Components 7/15/74 

6. Report on Cost and Marketability 7/15/74 

7. Final Evaluation Report 9/30/74 



III* FORMATIVE EVALUATION OVERVIEW*''^^ 



Of the fifteen program components, the six most relevant to -Instruction 
form the principal subject matter of the formative evaluation design* They 
are: Employer Support, Employer Utilization, Basic S.kills, Career Develop- 
ment Skills, Guidance and Instructional Systems* The remaining ccJmponetits, 



e.g. y Supplementary Programs, may be included as they relate to the instruc- 
tional program but will not be given priority attention. For each of the 
instrumental components the formative evaluation process will be organized 
to address the issues identified in the "Evaluation Plan for FY 197*4" and 
generally to provide the following information: _ 
1. An explication of the purpose, composition, organization, procedures, 
and operational strategies. — 



2. Evidence regarding the degree to which components are meeting stated 
objectives . 

3. Detailed information on the costs associated with each component. 
This information will be gathered by procedures to be discussed below. In 

t 

general, the formative evaluation is more flexible than the^^svmma^^ 
formally structured and less statistically sophistica^g^. ^^^he formative effort 
will result in the following reports: 



REPORT • ' DUE DATE 



1. Instructi^al Systems Design 12/31/73 
'-Employer Support Evaluation ^ * 4/15/74 

3.. Guidance Evaluation I ' 4/15/74 » 

4. Basic Skills Evaluation 4/1^/74 , 

5. Instructional Systems Field Test ' ' 4715/74 

6. Employer Utilizati^ Evaluation 5/ 1/74 — 

7. Guidance Evaluation II ' ' 6/ 1/74 

8. Career Development Evaluation 6/ 1/74 

9. >nstructional Systems Evaluation .7/ 1/74 

For each instructional unit except Guidance one evaluation report will be 
completed during the year; the Guidance tas^s have been split into two 
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reports. For Instructional Systems there are two additional reports because 
the evaluation staff is designing and field 'test ing , as welL as evaluating, 
that unit. All other components are being designed and field tested by the 
developmental and operational staff in each area* This outline of>t.the objec- 
tives and products of formative evaluation will be expanded in procedural 
and results sections below* 

IV* STUDENT POPULATIONS 
Four different student groups have been selected for analysis of the 
Career Education Program. Two of these groups are involved in the experimental 
program: 

1. ACE Group - (n=76) These students were in the Academy program last 
year for their eleventh grade experience and are continuing this year 
toward graduation. They are participating in the comprehensive 
program: all their school act;ivities -are conducted through the 
Academy* They will receive Academy for Career Education diplomas 
upon graduation this year. 

2. ACE~01ney Group - (n=76) These students were in the^Academy program 
Tor the first time this year, and their involvement is on a shared- 
time basis with Olney High School, a large secondary school in the 
Philadelphia publTcN^chool system. They participate in the core 
Academy program (Career Development, Career Guidance and Basic 
Skills), while receiving other courses and extracurricular activities 
at their home school. They remain on Olney's rolls and will receive an 
Olrvey diploma. This group was composed of tenth %id eleventh graders 

* 

at entrance* 
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Two additional groups of students were selected for the purpose of comparing 
the progress of students in the Acadeiny program with students in a traditional 
high school program. Ihese control groups were selected from the Olney High 
School student body: 

3/ Comparison Group - (n=28) These students applied for the Academy 
program, were accepted, but eventually declined to enroll. They 
were selected for comparison because they volunteered for a career- 
oriented program (evidencing a level of interest), ouu they passed 
the program's requirements. They were all eleventh graders. 
'4. Context Group - (n=81) These students represent a random selection 
of equal numbers of tenth, eleventh and twelfth grade students from 
Olney High School. They were selected td provide comparative data on 
the "typical** Olney student, 
or these four student groups, all but the ACE Group were selected from the 
Olney High bchool student body. The ACE-Olney and Comparison Groups were 
selected from among valunteers for the program (See Special Report, "Recruit- 
ment and Selection.") Th«f» Context Group was randomly selected from among 
their peers » Application for the ACE Groups was open to all public and non- 
public eleventh grade high school students in Philadelphia last year. These 
students were selected from a stratified random sample of the applicants; the 
group was intended to be representative of secondary students city-wide (See 
FY 1973 Evaluation Report 5*1.1, "Publicity and Selection Process*). Since 
these groups differ in their origin and do not fit into a traditional exper- 
imental design, extensive analysis of group characteristics will be performed. 
A precise analysis of between group differences, particularly with respect to 
dependent variable measures, needs to be accomplished before an analyses of 
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comparative effects can proceed. Presenting these pre-experimental analyses 
Js the principal objective of the "Pretest Results*' section below. In 
pursuit of this objective much descriptive information about the various 
groups will be displayed. . Before these results are discussed the instru- 
ments and procedures* employed will be described.,^ 

V. INSTRUMENTS AND PROCEDURES 
Each of the student groups described above was administered a series of 
instruments in a pretest-posttest design covering the 1973-1974 academic 
year. To date only the pretest package has been completed. In addition, 
the experimental groups have been administered various tests and surveys 
during the course of the year; these have occurred in several cycles. Results 
from the pretest package will be reported in the "Pretest Results" section 
below. Results from other measurement techniques will be reported in "Interim 
Formative Evaluation Results" sectipn below. The "Interim Summative Evaluation 
Results" section will incorporate any results having summative implications. 

Pretest-Post test Instruments 

The following instruments were included in the pretest-posttest design. 
They were administered to both experimental and control groups. Their 
primary use was intended to be in summative evaluation, although they have 
some formative utility. 



1. Comprehenive Tests of Basic Skills (CTBS) - This instrument measures 
traditional academic skills. The Reading and Arithmetic subtests were 
used, yielding the following scores: Reading Vocabulary, Reading s 



Comprehension, Reading Total, Arithmetic Computation, Arithmetic 
Concepts, Arithmetic Applications and Arithmetic Total. The 
instrument has been well developed and documented, but is subject 
to the usual insensitivities of standardized instruments. 
Career Maturity Inventory (CIII) - This instrument was designed to 
measure Career Attitude and a set of career competencies: Self 
Appraisal, Occupational Information, Goal Selection, Planning and 
Problem Solving. The Self-Appraisal subtest was not .administered. 

-This instrument has been well developed but not extensively 
, researched and documented. ^ 
Assessment of Student Attitudes Questionnaire (ASA) ~ This instru- 
ment has been designed by RBS staff to measure attitudes toward 

^several elements in the learning environment: Education in General, 
School Curriculum, School Resources, School Counseling, Learning. 
This instrument is still in the development stage, ^and extensive 
data are being gathered on its performance; technical reports are 
available. 

Student Demographic Data Questionnaire (SDQ) - This instrument 
was constructed by the evaluation staffs of all Experience Based 
Career Education projects to provide common data on basic charac- 
teristics. This questionnaire includes: Name, Sex, Birth Date, 
Race, Grade Level, Post Secondary Plans, Parents Education Level, 
Parent Occupations, Sending School Grades and Sending School 
Attendance. Many items require one-time administration only; 
selected items will be administered in the posttest. 
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Other Instruments 

Another series of instruments has been established for the purpose of 
assessing experimental students only. Although the results of these measures 
may have summatiVe evaluation implication, no comparable control group data 
would be available to establish a comparative perspective. These instru- 
ments have all been developed by the evaluation staff. Their primary 
intended* use was for formative evaluation; in some cases operational needs . 
were also accomplished through the evaluation activities. 

1- Employer Cluster Tests - These instruments are being designed as 
a series of tests of knowledge relevant to employer cluster learn- 
ing activities. They are to be administered as pre-and posttests 
for each cluster (academic quarter). These instruments are currently 
in the development stage, and no empirical data are yet available. 
Results will have both summative and formative relevance. 
2. Student Opinion Survey - This instrument is being developed as a 
common instrument by the evaluation staffs of the four Experience 
Based Career Education Programs. It is designed to measure student 
opinions concerning major program activities and objectives. The 
instrument is being subjected to extensive analysis for refinement 
purposes. It has been administered once, at mid-year, and further 
administrations are pending first results analysis.. The common 
instrument will be used both summatively and formatively. 
3- Parent Opinion Survey - This is the parent counterpart to the Student 
Opinion Survey; the information above regarding administration and 
refinement is pertinent here also. 



3.1 
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Employer Opinion Survey - This Is the employer counterpart to the 
Student Opinion Survey; the Information above regarding adralnlstra- 
tlon and refinement Is pertinent here also. 

Employer Checklist and Interview - This Instrument has been developed 
to monitor employer recruitment, orientation and management. It 
Is lii the process of first administration and will be readminlstered 
near the end o^ the year* 

Career Exploration Student Questionnaire - This Instrument has been 
designed to gather basic student reactions to employer ' learning 
activities. It is administered during the last activity session, 
and results are quickly fedback to the employer staff. 
Forms - The generic term is used to reference a fairly sophisticated 
set of basic data gathering procedures which have been developed 
and implemented by the evaluation staff. This evaluation activity 
has been detailed in Task Report 15A1. Develop Instructional 
Systems for Facilitating the Integration, Utilization and Communica- 
tion of Learning Resources * These systems have been developed to 
maximize computer applications. The forms designed and implemented 
Include! 

a. Student Needs and Interests Form 

b. Student Summary Sheets 

c. Learning Activities Descriptor Form 

d. Student Transcript 

e. Student Grade Report 

I 

f. Weekly Attendance Report 
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Pretest^^ Posttest Procedure . 

Figure 1 displays the testing schedule employed this year. During 
October, 1973 the Comparison and Context groups were administered all of the 
above instruments in one session; whereas, the ACE and ACE~01ney students 
were given the CTBS/CMI in one session and the ASA/SDQ in another session. 
One time period (4 hours) was the only release-time arrangement possible 
for Olney students. 

ACE and ACE-Olney students were tested on the CTBS and CMI during their 
orientation sessions before the opening of classes in September. The 
average data completeness for the CTBS was 91.45%; for the CMI it was 90.13%. 
Students missing during these scheduled sessions were eventually tested, but 
late scores have not been included in the analyses. ACE and ACE-Olney 
students were given the ASA and SDQ during their regularly scheduled 
Guidance Group Sessions after the school year had begun. The data complete- 
ness for the ASA was 82.00%; for the SDQ it varied widely by item but always 
exceeded 70.00%. The percentage of completeness warrants the assumption that 
these data represent the experimental groups in the Academy program. 

Comparison and Context group students were tested in a special session 
at Olney High School. In all, 183 students (57 comparison and 126 context) 
were scheduled for testing. Of these, 149 were actually notified by their 
homeroom teachers. Of the 149 subjects who were selected and notified, 110 
were in attendance for this testing session. This 26% absence rate indicates 
the possibility of a percentage nonrepresentativeness In the final sample 
attained. However, given the extremely short lead time for setting up the 
special testing session, and the lack of an overt ^reward for* attending, the 
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lostruzaentt 

CTBS - Cocprehensive Tests of Saslc Skills 
CM - Career Maturity Inventory 
ASA - Assessoent of Student Attitudes Scslt 
SDQ - Studtnt Deaosraphlc I>«ts Questlonnalra 

Groups 

ACE - 12th grsde Acsdexy students 

ACE ' Olney - lOth snd 1 1th "grade Acadeay students, in coopers tive 

prograc with Olney High School 
Coaparlson - 11th grade students who applied to Acadea; progrsa» 

ware accapted, but opted to not enroll 
Context • randooly selected 10th, 11th and 12th grade Olney Hl(h 

School students 

All pretests were adninlstered In I973» posttests In 1974. 

For each Instruacnt the "T** column indlcstes tlac In Months elapsed 
between pretest snd posttest. ' 



5. * indicates coopleted testing as of data of this report 



74% attendance rate is rather remarkable • The data seem to allow a reason- 
able degree of generalization to the original randomly constituted sample. 
All instTutftents .were administered in this special session* The data complete- 
ness for those students attending in most cases approached 100%* 

It was possible to establish good rapport in the testing sessions for all 
groups. There was nothing to suggest systematic between group differences due 
either to the administration procedure or to the motivation to complete the 
test instruments. All testing was accomplished Under standardized conditions 
by 'the evaluation staff. 



As can be seen in Figure 1 the times available for test administration 
were not optimal. Experimental groups were pretested earlier than contrcd 
groups, ctiiu Instruments could not be administered as a package in every cape, 
This was certainly not by design, but was reflective of the difficulties 
inherent in the testing of several hundred stu^nts on lengthy instruments 
whose scheduling required the joint planning of numerous agencies and indi- 
viduals. 

The first difficulty presented by the testing schedule is the possible 

non-comparability of pretest data between groups. This is important because 

establishing between group disparities on pretest dependent variable measures 

will deteririne the nature of the pretest-posttest gains analysis. The most 

serious element, the June testing for the ACE Group on the CTBS, was 

eliminated by an earlier decision to use last yearns pretest data for the ACE 

Group CTBS analysis. The use of last yearns data in this one case not 

'only equates the groups for age at t;lme of testing, but also sets up the 

analysis for one-year and two-year gains. In pretest-posttest analyses last 

yearns and this year's data will* be used to the fullest extent of their 

comparability. In the present analyses it is important to note th^%:he 

ACE Group results for the CTBS only are derived from FY 1973 pretest data. 

This ^ill leaves a one month discrepancy 4.n favor of the controls between 
r 

the experimental and control groups. Rather than incur the complexities and 
validity threats incumbent with score adjustments, this discrepancy has be^n 
accepted as a limitation. 

The second concern is one of intertest interval. It is important that 
the time expiring betwcijph the pretest administration and posttest administra- 
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tion of an instrument be roughly equal across groups • This is to insure 

comparability of gains as a function of time. Figure 1 also displays 

the projected posttesting times which have been staggered to attempt equality 

of intertest intervals. Precis^equality was not possible in all cases 
because pretest instruments were not always administered as a package, but 
posttest instruments will be so done. Scheduling priority was given to 
dependent measures in order of their perceived importance. Assuming a 
minimum of logistical impediments to implementing this schedule, analysis 
without serious limitations due to testing sequence should be possible. 

Other Test Procedures 

The pretest-post test instruments have been treated mare or less as a-^- 
package. - This section deals with instruments that function independently. 
Their forW and procedure is often less formal and fixed; they are also in 
different stages of development. These instruments will thus be discussed 
individually. 

Employer Cluster Tests . Delay in filling the instrument development 
staff position responsible for conducting this substantial effort in test 
construction has resulted in a revised implementation schedule.^ Knowledge 
tests for three clusters (Health, Manufacturing and Utilities) are planned fo 
trial administration at the end of the sixth quarter. These tests will 
then be administered as pre-and posttests by the evaluation staff at employer 
sites during the third quarter. It is anticipated that other cluster tests 
can also be developed during the seventh and. eighth quarters. 



student Opinion Survey . This has been administr^red at mid-year by 
the evaluation staff to students during their regularly scheduled 
Individuallz^xi^^rning Center sessions- A ^eaj^Sa^ administration of the 
revised instrument is anticipated. 

Pareift 9i>ln ion Purvey . This has been administered by direct mailing to 
parents* The response rate has been poor to date (about 40%). Efforts 
to retrieve more completed forms through individual followup are currently 
underway. A year-end administration is being, considered. 



Employer Opinion Survey , This is currently being administered through 

individual interviews of employer representatives by the evaluation staff. 

* 

A year-end administration under the same conditions is planned. 

4 

Employer Checklist and Interview . This has been packaged with the 
Employer Opinion Survey. 

Career E-xploration Student Questionnaire . These are adniiaisilTed by 
the Counselor-Coordinators during the^^^^Wt session of each employer learning 
activitiy. This pattern will continue throughout the year. * 



Forms. The forms system involves submission of basic data on learning 
resources available and student progress each quarter. These Constitute the 
foundation of the computer system. Explanation of the numerous procedures 
involved would be too cumbersome for the present report. For more discussion 
see Task Report 15A1. 
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Hypothesis Testing 1 '*'*•*• ■ 

As can be seen, the instrumentation'and consequent data gathering 
are substantial as well as principal evalaution activities. These functions 
provide much descriptive information for feedj^ack to employers, statf and 
others. They provide data ^useful in program development and management. . 
Several of the products are of operational use or have some worth in public 
information and dissemination..^ But it is the final analysis which really 
counts. This is an analysis of program effects as determined by the testing 
of prespecified hypotheses. These have been stated above in the "Summative 
Evaluation Overview". Basically, principal student effects will be deter- 
mined through rigouous statistical procedures employing the pretest-posttest 
package as a set of dependent variables- The independent variables will be 
alternatively entry level for gains analysis and type of prop,ram for com- 
parative effects. Other analyses utilizing student characteristics will be 
^o\iducted. Hypotheses relating to other effects do not have available 
structured statistical testing techniques, llather the informatioA from the 
summative package, other instruments, arid observations will b^ presented and 
discussed to argue degrees of success or failure. A more exacting discussion 
of proposed -analytic techniques may be found in Special" Evaluation Report 
3, "Revised Analysis Plan"- ^ , 

VI. PRETEST RESULTS* ^ 
In this section results and conclusions derived f orm ^e pretesting session 
of the pretest-posttest package will be presented. Results from the other 

instruments and procedures described above will be presented in subsequent 

< 

sections. 
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Table 1 presents several summary characteristics of the student groups 
ii^^ol\j;ed in the FY 1974 program. At the start of the academic year bSth 
experimental groups (ACE and ACE-Olney) contained 76 students. The context 
group (randomly selected) was roughly equivalent vwirdi^^Sl^^tudents , while 
the other control group (the comparison students /who had opted out of the 
program) was much smaller with only 28 students. The total number of sub- 
jects available for analysis was 261. 



Table 1 

Coopositlon of Student Croups 



^S^^^^ Croups 
Ch a r a c t e r 1 s 1 1 cTSin^^ 


ACE 


ACE-Olney 


Cocparlson 


Context 


Tot«l 


1. Size 


76 


76 


29 


81 


261 


2. Average Age 


17.0 


15.8 


16.3 


16»3 


16. A 

4 


3. Crade Level 


12.0 


10.5 


11.0 ' 


11.0 


11.2 


4» Prevl!ous School 
Z Attendance* 




91. f 


90.5 


89.7 


90.1 


5» Previous School 
CPA ** 


3.5 


3.0 


3.3 


3. A 


3.3 



ACE Group - Students originally recruited for FY 73 progratn. all equivalent 
of 12th graders, all In progran for second year. 

ACE-Olney Croup - Students recruited fnr FY ■'6 prosrss 1.-. =cc?cr*tl^.i wiJi 

Olney High School, grade equivalent split between 10th and 11th gradert , 
all in prograa for fir«t year. , 

Co«pariaon Group - Olney students who applied for ACE-Olney prograa, were 
accepted, but finally decided to not enroll. 

Context Croup - A rsndoo selection of equal numbers of 10th» 11th and 12th 
^ ^adera froe Olnay. no known exposure to progran, no intanded talectlon 
biaacSk 



* Data coopleteness - 84^. scale OX - lOOX 
**^ Data completeness - 96%, scale 1 ■ high to 5 - low 



In age, the ACE group was approximately one year older, on the average, 
than the other groups* Likewise in grade level, the ACE group was approx- 
imately one level higher than the others/ The ACE group consisted entirely 
of 12th graders. The ACE-Olney group was about evenly divided between 10th 
and 11th graders. The Comparison group consisted entirely of 11th graders. 
The Context group was divided 'among the three levels. 

Previous School Attendance 

Reported previous school attendance was uniform across groups and high 
(90%). It seems likely that this figure has been subjected to error some- 
where along the way, but it reflects school records as accurately as they 
could be analyzed. 

Previous School GPA 

Previous school grade point average (GPA) varied" slightly a^cross groups. 
On a 5 point sfcale with 5 being low, the ACE group averaged 3.5 (C-) , * • ^- 
the ACE-Olney group averaged 3.0 (C)^ and the controls fell roughly in bejtween 
Thi^ would indicate that the 12th grade experimentals have relatively poorer 
past school records than the 10th and 11th grade experimentals (p <^ .05). No 
other comparative differences were significant. Relationships to other 
measures and A^cademy performance remain to be drawn. 

■- • --^ - 

Race and Sex 

r 

With regard to sex and race, data were not gathered for the Context and 
Comparison groups. Table 2 presents figures for the ACE and ACE-Olney groups 
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Table 2 
Sex 4nd Rice Distributions 



Group 












Subgroup 


ACE 


ACE-Olney 


Total 




' t 


# 


Z 


# 


Z 


H«le 


34 


1 

49«' 


36 


56 


70 


52 


Female 


36 


51 


28 


U 


64 


48 


Blacky 


47 


67 


52 




99 


74 


White 


23 


33 


12 


19 


35 


26 


Black Hale 


21 


30 


26 


41 1 


47 


35 


llach fenale 


26 


37 


26 


41 


52 


39 


Vhlte HaIc 


13 


19 


10 


15 


23 


17 


Vhlte Fcoale 


10 


14 


2 


3 


12 


9 



The sex distribution was relatively equitable, although the ACE--01ney group 
had a disproportionately large number of males. The race distribution did not 
reflect equal representation; the percentage of Blacks in the 12th grade 
group was 67%, and in the 10th and 11th grade groups it was 81%. The total 
number of Whites recruited this year was very small (12). Subgroups overall 
ranked in order of size were: Black Females - 39%, Black Males - 35%, White 
Males - 17%, and White Females - 9%. It is apparent that the program is 
differentially attracting population subgroups; the factors involved in this 
phenomenon should be investigated* 

Parental Occupations 

Tables 3 and 4 present the distributions of parental occupations for the 
various groups. No major systematic differences between groups were observed* 
Preponderant categories for fathers were: Operative - 33%, Craftsman - 16% and 
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Laborer - 13%* The largest categories for mothers were: Housewife - 42%, 
Clerical - 18%, Operative - 13% and Services - 12%. For further analysis 
these occupational categories should be translated into a scale; at present 
they permit descriptive analysis only. 

Parental Educational Levels 

Tables 5 and 6 present the distributions of parental educational levels 
for the ACE and ACE-Olney groups. Most parents were reported to have com- 
pleted some orwail of secondary school. Since the levels have some scalar 
quality, a test for differences between the groups was made. The scale 
employed represented 8 educational levels from 1 = None to 8 = Advanced Degree 
The average educational levels of the ACE group were 3.91 and 3.,96 for fathers 
and mothers respectively; the ACE-Olney averages were 3.95 and 4.11. No group 
differences with regard to parental education were found to be significant. 

Post Secondary Plans 

All groups were questioned about their post secondary plans. Table 7 
presents this information. The two experimental groups did not seem to differ 
markedly with approximately 25% planning to immediately enter a vocatidn, 
over 50% planning further education, and about 10% anticipating job training. 
The ACE-Olney group was relatively more interested in 4 year colleges within 
"further education" by a margin of 35% to 22%. Of note is the wide disparity 
between the experimental and control groups. In the case of the latter, for 
Comparison and Context groups respectively, fuily 50% and 40% were planning 
immediate entry into a job, 42% and 48% were planning further education, 
and very few were anticipating anything else. From these distributions it 



28 



Table 7 ^ 
Fosc Secondar> Plans 
In Percent^ 



^"Vi^s^ Croup 
Category 


ACE 


ACE-Olney 


Coe^arlson 


Context 

■ 


Total 


1. Eeployoent 


19.4 


17r6 


50r<3 


39.2 


28.3 


2. Job Training 


10.4 


6,1 


4.2 


7.2 


8. 1 


3. Military 


7.5 


6.5 


0.0 


4.3 


5.4 




0.0 


0.0 


0.0 


0. 0 


0.0 


5< Vocational School 


14.9 


8.1 


4.2 


7.3 


9.5 


6. 2 yezr acadeaic 
coIle|e 


13.4 


6.5 


4.2 


1.4 


6.8 ^ 


7. 2 year vocational 
college 


3.0 


8.1 


4.2 


0.0 


3.5 


8. 4 year college 


22.4 


35.4 


29.0 


39.2 


32.0 


9. Part-tl»c work 


7.5 


6.5 


0.0 


0.0 


4. 1 


10. Other 


1.5 


3.2 


4.2 


1.4 


2.0 


Dlftca CospleCencss 


88.2 


81.6 


85.7 


85.2 


85. 1 



seems that, while all groups have a high incidence of planned post secondary 
education, the remaining control students are more interested in 4niraediate 
employment than the remaining experimental subjects. The plans of ACE and 
ACE-Olney students were more evenly spread over the categories available 
(possibly indicating more diversity of interest); the Comparison and Context 
distributions w^re biraodal within the employment and further education 



/ 



categories. 
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Reasons for Academy Enrollment 

Regarding reasons for Academy enrollment (Table 8) , the opportunity 
for Career Exploration was the largest single factor for both ACE and 
ACE-Olney students. Individualized instruction, choice of courses, smalle 
classes and the opportunity to move around the city were also indicated by 
substantial number of students. 



Table 8 

Reason for Acadeoy Enrollment 
In Percencs 



< 



Group 

Cacegory 


ACE 


ACE-Olney 


Total 


1. Scalier Classes 


11.7 


10.7 


11.2 


2. Career 

Exploraclon 


30.0 


51.8 


40.6 


3. Choice of 
Courses 


8.3 


12.5 


10.3 


4. Opportunity to 

Move Around City 


f— 

11.7 


8.9 


10.3 


5. Individualized 
Instruccion 


23.3 


8.9 


16.4 


6. Hake Naw Friends 


0.0 


1.8 


0.9 


7. Other 


15.0 


5.4 


10.3 


Data CoKplatenass 


78.9 


73.7 


76.3 



V 
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Career Maturity Inventory 

Tables 9 through 13 present the results of the Career Maturity Inventory 
. pretests. In all cases percentilej_were used for analysis purposes; thus 
grade level differences have been ^Iken into account . Each subtest table 
consists of three subdivisions. The first presents basic descriptive data 
on each group; the groups are arranged in order of the magnitude of their mean 
score. The second is an analysis of variance table which indicates the 
degree of statistical reliability with which the largest mean can be con- 

« 

sidered different from the smallest mean. The third section indicates the 
statistical significance of the remaining mean differences. In all cases the 
numbering of means reflects the order presented in the first'section of the 
table. 

Inspection of these results allows some interesting conclusions: 

1. In no case was the Comparison Group different from the Context 
Group (control groups) 

2. In no case was the ACE Group different from the ACE-Olney Group 
(experimental groups) ^ 

3. In most cases both experimental groups were superior to both 
control groups 

These findings indicated no differences between first year (ACE) and 
second year (ACE-Olney) students on career maturity factors as measured by the 
CMI. These findings also showed no .differences between students who applied, 
were accepted, but dropped from the program (Comparison) and a random selec- 
tion of students (Context). However, both of these groups were consistently 
lower than the experimental groups. This suggests that at least one con- 
sistent difference between students who stay in the program and other 
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ERLC 



L.ir»tM MaturiiN Invonturv rrvtcst 
AlliCude SluIi' fcrccncllc* 
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NO 


»W4n Dlff. 


1. A< ^ 




40. ^^ 


2h.0'J 




I, AU - Olney 


70 


39.ftl 


i7.90 


0.7^1 


3 Coaparlcon 


28 


26.93 


2S.57 


13.62 


1*. Context 


77 


26.39 


23.56 


14.16 



Analysis of Vtrlsnce 





Sim of <sotisrr9 


df 


Mean Square 


F 


Between 

Groups 


11086. 79 


3 


3695.60 


5.55 


Wtthln 
Groups 


158 36S.32 


238 


665.40 




TotaU 


169452 12 


241 


Confidence Level of 
■ F (3.238) - 99 89Z 



Tukey Test 
for Dlfterenccs letweeo Mean* 



Hesi Diff 


1 


2 


3 






14 16 * 


13.42 * 


0.54 




3 


13.62 * 


12.88 


Critical 


Vslue - 13,21 




0. /4 




* p < .05 



Table 10 

Career Muturlty Inventory Pretest 
O^cupJtional Infornatlon Subtest Percentiles 



Group!) 


N 


Mean t SD 


Mean DHf. 


1 ACJF - Olncv' 


69 


50. 16 


27 28 




1 ACt 


62 


46 57 


25 47 


3. 59 


3 Context 


76 


29.67 


21. 16 




4. Cospsrlsoii 


26 


28.62 


24.80 


21.54 



Analysis of Variance 





Sum of Squartt 


df 


■^an Square . 




Setween 
Group* 


21268.45 


3 


7089 48 


11.67 


Within 
Groups 


X39103.42 


229 


607.44 




Totals 


I6037I 87 


232 


Confidence l^ve 1 of 
r ( 3.229) - 99.99X 



ERIC 



TukeY Tttt 
for Differencea Batwten M«ans 



Mean Dtff. 


1 


2 


3 


U 


21.54 * 


17.95 * 


1.05 


3 


20.49 * 


16.90 * 


Critical 


2 


3.59 




* P 

V 



l<bU It 

c incr KJturuy Inv^-ntorv I'rctest 
Coal i>electii'n Subtest Percentiles 



CruupK 


N 


H«an 1 SD 


Mean Diff. 


I ALf - OIney 


70 


39 HO 


18.80 




t ACE 


66 


39.58 


20. U 


0.22 


3. Coaparison 


26 


25,39 


20.28 




<t ^Context 


69 


23 78 


16. 18 


16.02 



Analytla of Variance 





ituirr tt Squares 


^ df 


1 tte^n.SQuare 




hi. cween 
Crouph 


141 16.48 




47d5.49 


13.A3 


Within 
Croups 


•79551.24 


227 


350. A5 




Tc^t als 


93667. 72 


230 


Confidence Level of 
F ( 3,227) - n 99X 



Tukey Teat 
for Differences letvcen Means 





Mean Difl. 


1 




3 






16.0^ * 


15.80 * 


1.61 




3 


lA.Al * 


1A.19 * 


Critical 




2 


0 22 







Table 12 

Career Maturity Inventory Preteat 
Pianninj Subtest Percentiles 



Groups 


N 


Mean 


SD 


Mean Diff. 


1 . ACE 


65 


A4.97 


24. ?9 




2. ACE - Olney 

*7 


70 


A4.90 


21. OA 


0,07 


3 Covparisoil 


22 


33.32 


21.60 


11.65 


4. Context 
. t 


56 


27.57 


17. 79 


17.40 



Analysis of Variance 









df 


Mean Squsre 


F 




letwef n 

Groups 


12771. 19 


3 


A257.06 


9.22 




Within 
Groups 


96A50. 73 




A61.A9 






Totals 


109221.92 


212 


Conf idenca 
F ( 3.209 


Level of 

) - 99. 99X 



Tukey Ttat 
for Differanca. latwean Means 



Mean Diff. 


1 


2 


3 


A 


17. AO * 


17. 33 * 


5.75 


3 


11.65 


11.58 


Critical 


2 


0.07 




* p.< . 
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Table' n 

C40ber Maturity Inventory Pretest 
Problen Solving Subtest Percentile* 



Croup* 


N 


Mean / 


SD 


Mean Diff. 


1. ACE - Olney 


70 


A. 77 


21.61 




2. ACE 


66 




21. 61 


5.A8 


3. Coa()arl«on 


21 


28.57 


22.63 


21.20 


Context 


53 


28.32 


21. 31 


21. A5 



Analysis of Variance 





Sua of Squares 


df 


Mean Square 


r 


ftctvecn 

Groups 


17813.89 


3 


5937.96 


12. 76 


Within 
Groups 


9588^.56 


206 


&65. i*b 




Totals 


113698.^6 


209 


Confidence Level of 

F ( 3»206) • 99. 99X 



TuVtey Test 
for Wftercnce Between Means 



Mean DUf. 


1 


2 


3 




21.45 * 


n.97 * 


0.25 


3 


21.20 * 


15.72 * 


Critical V 


2 


3.^8 




* p. 



1 



students their age is a higher evidenced career maturity. This must be a 
factor at work in the recruitment and selection process, and merits further 
study. In percentile categories the control subjects were generally close 
t^o the lowest quartile while experimental subjects were close to the 
middle • 
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Comprehensive Tests of Basic Skills 

Tables 14 through 20 present the results from the Compr^^isive Tests 
of Basic Skills in the same format used above for the Career Maturity 
Inventory. The Reading and Arithmetic subtests were administered. Since 
scale scores were available for this instrument, they were use<^^ for analysis 
purposes to increase discrimination and accuracy. Grade equivalents are 
reported in the text for descriptive purposes. Since scale scores do not 
equate for age, the ACE group's pretest scores for last year were employed 
in this analysis. * - \ 

The statistical procedures demonstrated no significant differences 
between groups on the Arithmetic subtests. However, on each of the Reading 
subtests the ACE group was significantly superior to the Comparison group. 
The implications of this single comparative difference are not easy to draw, 
but it seems ciiBar that the ACE students upon entrance to 11th grade were 
better in reading skills than the stu^^^^^^ who applied but dropped one year 
later. The lack of other significant differences would suggest that any 
other group comparison^relating to basic skills could assume an initial 
equality of group achievement. 

Grade equivalent averages for each group are present^ in Table 21. 
As can be seen, most groups were functioning at the* 7th or 8th grade level 
on the average. Arithmetic scores were generally lower than Reading scores* 
All groups ranged on all measures from a low of the 3rd grade level to a 
high of the 12th grade level. From these low scores it is apparent that 
basic skills improvement is' a priority need for these students. 



CTIS Pnusc 





1 




SD 


Mm Nff . 


I. ACS^ 


70 


574.20 


71.72 




2. CoMtut 


11 


551.57 


9«.71 


22. «3 


3. ACZ - OXmmy 


73 


554.51 


7^.94 


29. «2 


4. Co«f«rl««a 


21 


529.4* 


73. » 


44.74 



I 





Sua mf S^r«« 


*t 




r 


Croup* 


52213. Ot 


3 


17427. «9 


2.43 


Within 
Crou{>* 


1711231.11 


241 


7112. 3f 




Totals 


1133514 94 


251 


Co«fli«oc« Uv«l of 

r ( 3 ,241) - 93.39X 



Tuk«7 TttC 
for Dtlfftr«oc« Mtwra Httaa 





1 


2 


3 




4 


44.74 • 


22.11 


15.12 




3 


29.(2 


(.99 


Crltlctl V«lut - 42. t9 


2 


22.(3 




• ». < .05 



^972-1973 pntcaC tcorts m*4 for this irouy to v^Mtt for if* 
^Ifftrtncts; all oth«r scorM •f 197>>1974 



T«blt 15 
CTIS Prtttat 
lUi^lnt C«iyrthtns^ik.5calt Scortt 



Ctovf* 


N 


Mm 


SD 


Mftsn Dlff. 


1. ACE* 


70 




•9.4( 




2. ACI - Olnty 


73 


539.18 


79.71 


23.11 


3. CoDttxt 




^22. 33 


90.00 


40.73 


4 Coii^irlson 


2S 


$12.00 


95.05 


51.04 



Aul/sia of V«rl«fict 





of Squcrtt 


df 




T 


lettfo^n 
Croups 


•2971. 75 


3 


27(57.25 


3.41 


Wilhln 
Croupt 


19025(7.(4 


241 


7(71.(4 




Totals 


1915539.41 


251 


ConiUmtiCm I 
T ( 3,241) 


of 

f 98. (OZ 



Tvkty Ttat 
for Dlfftr«dc« l«tw««n MftAna 





1 


2 . 


3 




4 


51.05 * 


27.11 


10.33 




' 3 


40 73 


17.55 


Critical Valu* - 44.31 


2 

\ - 


23.11 




. *p.<.05 



1972*1973 pr«t««t ccorat utad for this irouf to a^uata far aga 
4iffar*ncaa( all othar acoraa ara 197V1974. 



T4kl* U 
criS FrttMC 



36 





M 






Mm JAtt. 


1. KCX^ • 




541. 04 


•7.53 




2, ACE - Oloty 


73 


53^.21 


71.27 


2% » 




tl 


534.25 


t7.5I 


3i.il 


4. C«^*il«oo 


21 


517.75 


t3.41 


50. 31 



tjitlyl* of Vsrl«Bc« 





bus of S^uArtt 


41 




f 




44>49.44 


3 


22114. 5^ 




UUhln 

Croup* 


1771244 00 


24t 


V. 7U2 12 






1137595.44 


251 


^ CoofldMC* L«v«l of 
r ( 3.241) - 97.25X 



for Dlfftr««c« I«tveta Mean* 



Hc«n Dlff. 


1 


2 


3 




4 


50.31 • 


21 44 


1«.50 




3 


31 tl r 


2 t4 


Critical V»lu« - 42 74 




21 t5 




• f . < .b> 



'l97Z-l»73 prttMt score* for thU group to Mu«t« for age 

<ltf£ertncts. all other »core» are 197>l974 ^ 



CTiS Prctttt 
ArlthMtlc Copulation Scalt 5c&r«* 





N 


Mas 


SD 


)taan mff. 


1 Crouf ■ 

1. ACE^ ^ 


71 


ill V 


74.51 




2 Covfarlton 


28 


510 57 


10 01 


1 40 


) Contaxt 


10 


502 14 


93 ^9 


9.8J 


4 AC£ - Olney 


73 


494 11 


71 9) 


15 14 



Analytic cf Vxxlance 





Sua of Squares 


df 


Mean S<)uar^ 


T 






9751.41 


3 


J250 -9 


0 


49 




W1t>iln 


1442147. 40 


241 


fc^Ul.24 










144lt99.0» 


2S1 


Confidence L«v«l 
F ( 3,242) • 


of 

Not Co«9Ute4 



Tuk*y Teat 
for Wffereoc* letweto M»a«« 



Ma«" Pi"- 




Crirical Value • 41,03 



p. < .05 
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^1972-1973 pretcet acotea uaed for thit |rou|i tw equate for age 
4U(«renctt. •Jl other ecore* are 1973-1974 



t4klt II 

crts rr«tMc 

AriehMClc Co«c«ft« Sc«U Sctrtt 





M 






MiM DUf. 


1. AQl 


71 


531. »7 


10. 25 




2. CM^trlto* ^ 




~ 531.25 


74.17 


0.72 


) CoaCcxc 


•0 


5U.I} 


I*. 77 


17. U 


4. ACE - OUuy 


73 


513.M 


79. 1 1 


It. n 





t\m of Stuart • 


4t 




T 




tvtwccn 


11469.41 


3 


61 St. 54 


0,94 




Wllhln 
Croupf 


U}14ft7.U 


241 


6571.50 






Tot sit 


l(4993ft.M 


251 


CoofId*ac« Uvtl of 

T ( 3,2*8) • Not Coiirut** 



Xvkmy Tt«c 
for DIfftrt«ct I«Mt« Mc«n« 



K«4n DLff 


1 


2 


3 




4 


U.31 


17.59 


1.17 




) 


17 14 


14.42 


Critical 


VtlM • <>1.03 


2 


0.72 




* < .05 



^1972-1973 pretest acor^i uaad ftr thli »rou> to •fu»t« for tf 
41ff«r«ncc«, all other acoraa att 197)- 1974 



Taklt 19 
CTtS Frataai 
AflthMtlc Apfllcatlona Scala Scores 



Grouva 


N 


Htan 




Itean Dlff. 


1 

1 ACt 


71 


523 27 


87.65 




2, ACE - 01n«y 


72 


517.03 


12. t9 


4.24 


3 Co«tax( 


10 


50* 14 


85 80 


14 n 


4. Cottparlaon 


2t 


4tl 57 


iU.51 


41. 70 



Analyali af Varlanc* _^ • 





Sub af S^uarat 


it 


, man S»)iAt* 


T 


irtwvrn 
Croupi 


373tl.44 


3 


12442.49 


1.44 


Within 
Croups 


2073692.20 


247 


4315.52 




Ivtnla 


2111080.44 


250 


Confld«nc« Laval of 

f ( 3,247) - 74,071 



Tukar Tatt 
f«r Ulffaranc* Mtvaaa Mtaoa 



Haan Dlff. 


1 


2 


3 




4 


41.70 , 


35.44 


27.57 




J 


14 13 


7' 41 


CrltlCA^^alu* • 46,40 


2 


4.24 




• p. < .05 



1972'1973 prataat scarta ua«4 for thla (rouy to a^uata for •!« 
41f(arancaa. all oth«r acora* ara 197>*1974. 



Tab la 20 

CTIS Pracaat 
IrUhaatle Total Scala Seora* 



Croup • 


N 




SD 




1. ACI^ 


71 


517,06 


77,04 




2. Coaparlton 


2t 


504. )4 


74. Ot 


12.52 


3. Coataxt 




503. 3t 


94. tl 


13.61 


4. kCt - Olnay 


72 


496.36 


76.63 


11.70 



AnAljtla ot Variance 







SuK of S^xurea 


df 


)teazi S^uar* 


r 






IcCvcen 

Croups 


32303.42 


3 


10767.51 


l.Ot 






WUhln 
Croups 


2467073.27 


247 


9961.15 








Tot,al$ 


2499374.69 


250 


Coafld«ac« Uvel of 

r ( 3,247) • Mot C<»puta4 



for DlffartecX ISfwaan Haxna 



N.S 



Haan Dlff. 


1 


2 


3 




4 


U.70 


6.13 


5.02 


3 


13.66 


1.16 


Critical Value, - 43.60 


2 


12.52 




* p. < .05 



1972-1973 preteat acorcf uaad for thla jroup to e^uata for tga 
diffcren.«t, •] i othar acorta are 19^3-1974. 



Table 24 
CTBS Pretest 
Reading and Arlthnetlc M«an Grade Equlvalenta 



^^\,t;roup 










Total 


ACE^ 


ACE-Olney 


Cosparlton 


Context 


Teat 










RftAdlnt 












Vocabulary 


9.2 


8.5 


7.9 


0.6 


8. 7 


Co&p rehenalon 


1.7 


8.1 




7.5 


8.0 


Total 


8.8 


8.3 


7.5 


1.3 


8.4 


Azlchacdc 










7.6 


Coopucadon 


7.8 


7.4 


7.6 


7.5 


Conctpca 


8.3 


7.7 


7.9 


0.0 


8.0 


A;)pllcaclon 


7.8 


7.5 


8.4 


7.5 


7.6 


Total 


7.9 


7.4 


7.5 


7.4 


7.6 



1972-1973 pretest scores ^cd for ihU group co equate for a^o dlffcrancaa. 
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Assessment of Student Attitudes Survey 

The Assessment of Student Attitudes Scale is a measure which is currently 
being given much developmental attention by the evaluation staff. It is 
included in this section because it is a p^rt,of the pretest-posttest package 
aimed primarily at suinmative evaluation. 

The Assessment of Student Attitudes Scale was designed specifically to 
measure student attitudinal dimensions in several areas central ft o the instruc 
tional process . These areas have been designated as subtests wrth separate 
scores as follows : v. 

1. Education in General 

2. Program Curriculum 

3. School Facilities 

4. Program Counseling 

5. Learning in General 

The instrument is intended to measure student attitude toward the school pro- 
grain with reference to each of the areas listed above. The instrument is gen- 
eralized in content and phraseology to be applicable in public schools as 
well as alternative or experimental programs. Extensive reliability, validity 
and discrimination value studies are currently being undertaken. 

Tables 22 through 27 present the results from pretest analyses of each^ 
ASA subtest. The tables have been formatted in the same manner as those 

V 

reported above for the Career Maturity Inventory and the Comprehensive Tests 
of Basic Skills. Dat^ are reported as raw scores reduced to a mean on a 
scale from 1 (Isi^^ to "5" (high). As can be seen from these results, only one 
difference between groups was found to be significant; that was between 
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T«bl« 22 
ASA Prcceat 
Education Subtest R«w Scores 



Groups 


. N 


Me«n 


SD 


Hcan Dlff. 


1. ACE 


5ft 


3.49 


.55 




2. ACE - Olnty 


52 


3.46 


.64 


03 


3. Coapsrlson 


28 


3.42 


.13 


.07 


i*. Conteit 


80 


3.41 


■J 

.64 


.08 



Aatlysls of Variucc 





■'T-^ 


Sua of Sqiur«s 


df 


Me«n S«usr« 


F 




Between 
Croups 


2555.14 


3 


851.95 


0.2O 


N.S. 


Withti 
Croups 


187698. 15 


21^r 


4187.26 






Tot«U 


1902^3.98 


213 


Confidancc Level of 

F ( 3,212) - Not Co«pu 



for Difference lecveeo HeAns 



>k«n Dlff. 


1 


2 


3 




4 


.08 


.05 


.01 


3 


.07 


.04 


Critical Value - 0.35 


2 


.03 




* f. <.05 » 
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Table 23 
ASA Pretcat 
Progr** CurrlculuM Subtest Kaw Scorea 



Croups 


N 




SD 


tteu Dlff. 


K Aa ^ 


56 


3.4t 


.60 




~T 

2. Coaparlaon * 


21 


3 46 


.68 


.02 


3. Contaxt 


80 


3.41 


.59 


.07 


4. ACE - Olnay 


52 


3.31 


.ftl 


.17 



Analysis of Variance 





Sua of Squarca 


df 


Ktu Souarc 


F ' 


Between 

Croups 


1631.54 


3 


2879.51 


! 

0.77 j 


Within 
Croup* 


792636. 11 


212 


3738.85 




Total* 


801274.65 


215 


Confidence Level of 

T ( 3.212) - Not Cooputed 



Tukay Test 
for Dlffersnc* Itcvean Haans 



ERIC 



H«an Dlff 


1 


2 


3 




4 


17 


.15 


.10 




3 


.07 


.05 


Critical Valut - (T. 33 


2 


02 




* p. < ^05 * ^ 



41 



TabU 24 
ASA Precctc 
Pro|r«a lUioureaa Subceac R«w Scora* 





N 




SD 


He«n Dlff. 


I. ACE 


\ 


3.64 


.63 




2 Concexc 


to 


3.36 


.59 


.28 


3. CoBparlton 


28 


3.32 


. 74 


.32 


4 Aa - 01n«y 


52 


3.25 


67 


.39 



ADAly«l« of VAriance 





Sua of Square* 


df 


1 haan Square 


r - 




46690.93 


3 


15563.64 


3.78 


Withlr 


873276,61 


212 


4119 23 






919967.54 


215 


Confidence Level of 
F ( 3,212) - 98.56X 



Tukey Teac 
for Difference Batv*«n Means 



Hean Olff 


1 


2 


3 




.39 * 


. 11 


.07 


i 


.32 


.04 


Crit i.jtl 


I 


.28 




* P < 



TAble 25 
ASA Pretest 
fro|rsB Counaellni Sut>ceac Rav Scores 



Croups 


N 


Mean 


SD 


Mean Dlff. 


1 ACE 


56 


3.24 


88 




2. Concexc 


80 


3.00 


83 


.24 


3 ACE - Olney 


52 


2.t6 


91 


36 


4. Coaparlson 


28 


2.86 


.81 


.38 



Analyala of Variance 





Sun of Square* 


df 


Mean Square 


F 


Group * 


47223.23 


3 


15741.08 


2.Q9 


thin 

(\Ti V»p'. 


1597083 73 


212 


7533.41 


Level ci 

) - 8V.74Z 


Tot .1 1 


1644306.96 


215 


Confidence 
r ( 3,212 



Tukay Taac 
for Difference Between. fVana 



ERIC 



.38 



Critical Value - 0.46 
* p. < .05 



T*bU 26 
ASA Prattit 



Croupt 


N 


H««n 


to 


Hft4n Ulff» 


1. AC£ 


S6 




.63 




2. 'Ad - OUey 


51 


3.(6 


.63 


.03 


3. Co^«riioa 


21 - 


3.« 


.73 


.06 


4. Conc«xt 


•0 


3.63 


.67 


.06 



Arulyaii of Varisnct 







df 


HtaD Square 


r 




Between 
Croupa 


1727.09 


3 


575.70 


0.13 




Within 
Croups 


910245. 40 


211 


4313.96 






Tot«ls 


911^7 2. 4» 


214 


Confideocc 
T ( i,2ll) 


Uvcl of 

- Mot Co*put«<l 



Tuk«Y T«it 
for Difference Sctvaan Mtana 





Me^ Dl££. 


1 


2 


3 






4 


.06 


.03 


.00 




N.S. 


3 


.06 


.03 














Critical Value • 0.35 




2 


.03 




* p. < .05 



Tibia 27 
ASA Pr«taft 
Total Hew Scorcf 



Croupa 


N 


Mean 


SD 


Mttn Dlff. 


1. ACE 


S6 


3.51 


.49 




2. Context 


10 


3.36 


.53 


. 15 


3. Coaparlton 


28 


3.34 


.64 


. 17 


6V ACE,- Olney 


52 


3.29 


.54 


. 22 



Analjraia of Varlanca 





Sija of Squarea 


df 


Mean SquAra 


r 


Between 
Groups 


1465 2.05 


3 


4184.02 


1.71 


WUhin 
Group«i 


606434.72 


212 


2860.54 


Level of 
2) > I3.35Z 


Total* 


621086.77 


215 


Conf idenct 
T ( 3,21 



Tuk«y TaaC 
for Difference Ittvaan Heana 



ERIC 

hfiiinniimrfTuma 













Mean Dlff. 


1 


2 


3 




4 


.22 


.07 


.05 






.17 


.02 






/ 






Critical Value • .35 


2 


.15 




* p. < .05 h 



the ACE and ACE-^Oloey groups on the Attitude Toward Program Resdlirces Sub- 
test. All groups were found to be not different on all other subtest 
measures. It is apparent that all student groups are similar with regard 
to attitude toward school as measured by the RBS-designed Assessment of 
Student Attitude Scale. 

Summary 

This section has presented information on characteristics af students 
attending the Ac^'demy for Career Education. There were two groups of students 
attending programs at the Academy: the ACE group and the ACE-Olney group. 
The ACE group were 12th grade students who received their full high school 
experience at the Academy; the ACE-Olney group were 10th and 11th grade 
students who participated only in the core Academy program. Two additional 
groups *of students were selected for the purpose of comparing the progress of 
Academy students with students in traditional high school programs. These 
two groups were selected from the student, body at Olney High School; the 
^^roups were a Comparison group and a Context group. The Comparison group 
was composed of 11th grade students who applied for and were accepted for 
the Academy program but declined to enroir in the program. The Context 
group was composed of a random selection of equal numbers of 10th, 11th, and 
12th grade students. 

Four instrifments were administered to the four groups of students. The 
instruments were the Student Demographic Data Questionnaire (SDQ) , the Compre- 
hensive Tests of Basic ^Skills (CTBS), the Career Maturity Invento^ry (CMI) , 
and the Assessment of Student Attitudes Scale (ASA). 
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the control groups were Occupational Information, Goal Selection, and Problem 
Solving. The two other subtests of the CMI , Attitude Scale and Planning, 
presented inconclusive results with the ACE group superior to the Compari- 
son group on one and the ACE and ACE-Olney groups superior to only the 
Context group on the other. The significant differences on the three scales 
seem to be a factor in students' decisions to participate in the Academy 
program and warrant further investigation. 

The Comprehensive Tests of Basic Skills (CTBS) were also administered to 
all experimental and control groups. No significant differences between 
groups were demonstrated on the Arithmetic subtests. On each of the Reading • 
subtests, the ACE group was significantly different only from the Comparison 
group; while it seems clear that the ACE students upon entrance to 11th 
grade were superior in reading skills to students who applied and dropped, 
implications of this single comparison are difficult to draw. Other group 
comparisons support the contention that the experimental and control groups 
are equal in achievement on other basic skills. 

The Assessment of Student Attitudes Survey (ASA) was administered to all 
groups and revealed no differences between the groups in their attitudes 
toward education, program curriculum, program counseling, or learning. The 
only difference between groups was between the ACE and ACE-Olney groups 
in their attitude toward program resources; the ACE students appear to be 
more positive in t^eir attitude toward program resources. This difference 
may be attributed to the greater length of time the ACE group has been 
enrolled in the experimental program. 

The experimental and control groups were equal on many of the instruments 
administered. On the CTBS, all groups were statistically similar with the 
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The Student Demographic' Data Questionnaire (SDQ) provided information 

regarding the background characteristics of Academy students and their 

counterparts in the control groups. The ^tudents selected seem to 'be 

'representative of an urban population. There appeared to be no between-*group 

differences for previous school attendance or for parental occupation • 

Between- group difference.s were found for previous grade point averages 

(GPA) of the groups, post secondary plans of the groups and the racial and 

sexual compositions of the groups. The ACE students had a GPA of C-, the ACE- 

Olney group had a GP^A of C, and the control groups averaged in between the 

two experimental groups* All four groups showed a high level of interest in 

post secondary education. Sex and racial data were collected only for tUe ^ 

Academy groups. The control group students were more interested in immediate 

post secondary employment than were, the experimental groups; the experimental 

groups also showed a greater diversity in the post secondary planning than did 

the control groups. The ACE group was approximately evenly divided between 

Males and Females; 70% of the ACE group was Black and 30% was White* The 

ACE-Olney group was 55% Male and 45% Female; 80% of this group was Black and 

20% was White. The Academy program seems to be attracting significantly more 

Blacks than \^ites. If this trend continues the Academy program will have 

served members only of one population sub-group. The uneven racial composition 
snoula be studied further. , 

The Career Maturity Inventory (CMI) was administered to all experimental 
and control groups. Three subtests of .the CMI revealed distinct differences 
between the experimental and control groups. The subtests on which the ACE 
and ACE-Olney groups were equal to one'knother and significantly superior to 
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exception that the ACE group was superior to the Comparison^group on Reading, 
subtests. The ASA results indicated that all groups were similar with the 
exception thar-fche ACE group was superior to the ACE-Olney group on the 
Attitude Toward Program Resources subscale. 

The CMI and background characteristics obtained from the' SDQ did reveal 
some intergroup differences that warrant further study. The ACE and ACE- 
Olney groups were superior to both control groups on mos t^^$ub tests of the 
CMI. And the program seems to be differentially attracting Black and White 
students with Blacks forming a large majority of^the students. Thus, both 
career maturity and race seem to be factors which differentiate between 
students who display interest^nd enroll froiji those who display interest and 
don't enroll in the Academy programs. Reasons for these differences necessi- 
tate further study. 

. VII. COMMON INSTRUMENT RESULTS 

Four instruments have been developed in a cooperative effort among the 
evaluation staffs of the Experience Based Career Education (EBCE) Programs 
and the evaluation representative of NIE's Career Education Program staff. 
Three of these instruments, Student^ Parent and Employer Opinion Surveys have 
been developed to investigate program effects. The fourth, a Program 
Administrator Questionnaire is mainly a structured format for collecting 
information descriptive of project conduct. All common instruments have as 
their basic objective the consistent collection of useful information across 
projects. The goal of this activity is to maximize the research utilization 
of the Career Education Programs. The four instruments have all been developed, 
but each is in a different stage of implementation as described beldw". 
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Program Administrator Questionnaire . 

This instrument has just reached the final stages of design and has 
not been administered in time for inclusion of results in the present report, 

Employer Opinion Survey 

This instrument is currently being administered by the evaluation 
staff. Since last year's experiettc^ demonstrated that mailed questionnaires 
of this type were hard to retrieve, the administratio^/fs being done by 
personal interview* This process could not be completed for inclusion in 
the present report. 



Parent Opinion Survey 

This instrument was mailed to parents with stamped, self-addressed 
return envelopes. The response to da.te has not been encouraging (about 40%), 
but follow-up procedures are underway. No riesults were available for analysis 
in the present report. 

Student Opinion Survey 

This instrument was administered to all students during their Individual- 
ized Learning Center sessions the week of January 21, 1974. The data complete- 
ness averaged about 85%. The itl^ns^ from the survey have been divided into 
subsections which will serve to organize the presentation of results. 

Opinions of Program . This section includes Questions 1-10, 21 and 23-29; 
all of these items are scored on a scale from 1 = low to 5 = high. In items 
1"1(X and 21-24 the scalar extremes were '^Definitely NO*' to "Definitely YES"; 
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If 

for 25-29 they were "Poor" to "Excellent". Table 28'presents the results 
from these items. It is apparent that student opinions were fairly high 
with a substantial range across items. Items that were particularly high 
(Overall Mean ^ 3»99) concerned general appeal and interest value of the 
program, the opportunity to progress at one's own rate and the learning ex- 
periences. Items th^t were relatively low (Overall Mean ^ 2.99) were all ' 
"time choice" questions: student doA^t feelthat" they have much choice in 
determining the amount of time spetVCjjiX^ar ious activities. This finding is 
reasonable since the duration of activities is fairly fixed. The c^ 
which activities one enters is open, and this is reflected by the mu< 
higher mean (3.49) when students v*are asked if they ^ad enough choice in 
selecting the types of employer/resource sites visited. Of interest is the 
fact that in each item the ACE Group had relatively lower opinions than the 
ACE-Olney Group. The overall mean difference for these items was 3.90 vs. 
3.32 for ACE-Olney and ACE respectively. It would appear that the twelfth 
graders' general opinion of the program was substantially lower than the 
tenth and eleventh graders' at the time of testing. 

Vocational Attiti^de . This brief section includes Questions 11-15, all 
of which wWe-^ciate^on the "Definitely NO" to "Definitely YES" scale. Again 
1 = low and 5 = high. The results are presented in Table 29. Scores were 
. generally high, but two of the items were low (^ 3.54) for this group of ques- 
tions. Students did not appear to think that most people get much satisfac- 
tion from their work. They also were relatively down on money asy^ motivator ♦ 
Interpretation of these findings involves value judgements. In riU^ c^ses 
the ACE-Olney scores were again higher than the ACE figures. 
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^ Table 28 
Scvidenc Opinion Survey Preceac 
Opinion oJ^kftSS^^ 



Item 


n 


Total 

Hean 


ACE 
Mean 


Mean 


1* Have you liked atcending the Careet 
Education Prograii? ^ 


113 


3.99 


3.86 


4.11 


2. If you had ic to do over again. ^o 
you think you would decide co Ri<r- 
dcipace In che Career Education 
ProRrata? 


Lli. 


J . jU 




3*92 


3. Have che activities available in 
Che Career Education frograo been 
interesting to you? 


113 


4 .03 




4 ♦ 43 


4. In the Career Education Progras 
have you felt that you covild 
prci^ress at your ovn rate? 


112 


4.37 


4.05 


4.70 


5. Have you seen such of a relation- 
ship between your activities in 
che learning center and the careers 
^ou have learned abouf^ 


112 


3^29 


3.03 


3 .56 \ 


6. Do you get enough feedback about 
hov well you are doing in che 
proRran? 


112 


3.17 


2. 88 


J .mO 


7. Have you had enough choic^in 
deciding tht asount cf else/ 
you •fen'^ j»r ^r<oiov*r sices? 


109 


Z.b/ 


2. 39 


i. .70 


8. Have ycj had enough choice in 
deciding chV aoount of cine you 
spend in learning academic 
subjects^ 


111 


2.99 


2.88 


3.U 


9. Have you had enough^ choice in 

deciding what you do at ecployer/ 
resource sites'' 


112 


2.77 


2.52 


3.04 


10. Have you had enough choice In 

aeleccing the types of employer/ 
resource sites you visit? 


111 


3.49 


3.23 


3.76 


21. ifirough your experiences in the 
Career Education Program have 
you learned a lot about oppor- 
tunities for the fucure? 


112 


4.26 


4.10 


4*43 


23. Would you say the Career Education 
Prograci has helped you form career 
pUns? 


110 


3.89 


3.71 


4.07 


24. Wo'jld yoM say you*ve learne<i a lot 
while atcending the Career Education 
Pr'^j^ran'^ 


111 


4.04 


3,84 


4.24 


25. Huwwcll orxaalzcd And cour^Jln ito'l 
do you think the Career Education 
Program h.is been' 


111 


3.40 


3,02 


3.80 


26. How would you rate the p'^nrr.il 

qt^allty of 'the Cnretr Educ;itlon 
'\ Pro^r/im <taff' 


112 


3.90 


3.62 


4.20 


^z). How would you rate the personal 
cotmselinR avallible In the 
Career FduC'itlon PrOffim' 


110 


3.65 


3.21 


4.13 


28. How would vou rate the career 

counseling available in the Career 
FduoitJon Procram' 


111 


!3.68 


3.19 


4.19 


l^. Hov would you ratf the gencril j 

quillty of the C.ireer Fducotlon i | 
Program erpploycr/resources vou'vc i 
worVi d vltK' n? 'T 75 


3 50 


4.0? 


" A v< r , 1 s 4 li I ^{'1 1- VI L-J_i2_ 


3 . 'i<i 



Mc.mn on acali* from 1 - low to 5 • hlKh 



V 
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Ttble 29 
Scudeoc Opinion Survey ?rec«ac 
Vocational AcclCude 



Iced 

»> 


tt 


Tocal 
He an 


ACE 

Mean 


ACE-01ne> 
Mean 


11. Do oosc people receive ouch 
•«clsfactlon froa chetr work? 


108 


3.54 


3.55 


3.53 


12. Do you chink chac tf a person works 
hard enough, he can achieve anyChlnK? 


111 


4.41 


4.26 


4.56 


ly. Do you chink chac che oain reason a 
person works Is co earn enough ooney 
to live' 


111 


3.39 


3.14 


3.65 


14* -In general, /are you looking forward 
Co worklnj^ in a jot> ^ 


112 


4.50 


4. '31 


4.70 


IS. Do you Chink you have cuch choice of 
occupaclons' 


' 112 


4.15 


4.00 


4.31 


Averages 


110.8 


4.00 


3.85 


4.15 



Means oo scale froa 1-low Co 5»hlsh 



Opinionsc^EtHployer/Resources , This section included Questions 16-20, 
all of Which werkagarn rated on the "Definitely NO" to "Definitely YES" scale 
The results are^ presented in fable 30, Student opinion was again generally 
high* Student sense of ^welcome at employer sitds was especially good 0> 4.00) 
One area, feedback from employers to students, was very low «^ 3>00)t In 
e.acH' case ACE-Olney mean scores were higher than ACE Group scores. 



Table 30 




Studenc Opinion Survey Pretest 
Oplnlona of Enployer/Resources 



Itea 


n 


Total 
Mean 


ACE 
Mean 


ACE-Olney 
M^an 


16. In general, were the caployer/ 
resource personnel*lnvolved In 
the Career Education Prograa 
aware of your needs 'and 

_ Interests' ^ 


112 


3.35 


3. 19 


3.52 


17. In general, at cKtloyer/ 
resource sites did you get 
to actually do things, rather 
than Just .listen? 


111 


3.23 


* 2.97 


3.51 


18. In general, h^ve the eoployer/ 
resource sites you*ve visited 
H been interested In the Career 
^ Education Program'^ 


111 


3.76 


3.47 


4.08 


19. In general^ have you felt 
welcoae at the enployer/ 
resource sltes^ 


110 


4.00 


3.77 


4.24 


20. Do tiost of the esploycr/ 
resource sites you have 
worked with t€c you know how 
you're progressing^ 


110 


2.55 


2.40 


2.71 


Averages 


110.8 


3.41 


3. 16 


3.61 



Means on scale frotx 1»- loV t(i 5 ■ high 



51 



Reasons for Entering the Career Education Program . This secti'bn was 
comprised of a single, multiple part item. Each part represented a projected^ 
reason for entering and asked for a rating on the 1 to 5 scale from ''Not 
at All Important" to "Extremely Important". The results are presented in 
Table 31. Two of the projected reasons were accorded a very high (^4.00) 
importance; they were wanting to learn about careers and prepare for. a job. - 
Two others were rated low in importance (<3*00) ; they were boredom with 
school and anticipation of an easy time in Career Education. • -No notable- 
between group differences were observed. 

_ • Tible 31 _ . , , 

Student Opinion Survey Pretest 
XeA«ons for Entering the Career Education Pr-ogri« , ' r 



Itea 


a 


,_Iotal 
Kean 


ACE 
Mean 


ACE-Olney 
Heano 


30. ){ov iaportant was each of the 
following factors in deciding 
to Join the Career Education 
Prufraa? 

a. I wanted aore fre?d^^o/ 
Independence 


112 


3.52 


3.50 * 


3.54 


b, I wanted to choose my own 
leaminjt style 


111 


3.79 ~" 


^ 3.95 


3.62 ^ 


c. I wanted to learn about 
careers 


112 


4.29 


4.1^ 


4.41 


d. I didn't like ay previous 
school 


112 


_ 3.13 


3.41 


2.81 


e. I wanted to prepare for a 

Job ' 


112 


4.13 


J. • 

4.00 


4.28 


f. I was bored with school 


111 


2.87 


2.S18 


2.75 


g. I heard the Career Ed- 
ucation Proxrao was easy 


III 


2.d5 


2.00 


z 

2.11 


h. Other (specify) 


29 


3.90 


4.40 


3.63 


Averaj^es 


101.3 


3.46 


3.55 


a. 39 



Means on scale froo 1-low to 5-high 

Opinion Comparative to Traditional Schools . This- section included " 
only three items, Questions 31-33. They'were again on a' five point scale, 
but the poles were "Much Less" to "Much More". The results are presented 
in Table 32. Scores on these items were ' exFremely h&gh (>-3.97) evidencing 
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TAkIc 3 2. 
Student Opinion Survey Pretest 
Opinion Co«p«r«tlv« to Ir*dltloiul Schools 



Item 


n 


Total 
Mean 


ACZ 
Mean 


ACE-Olaey 
Mean 


31. In cooparison with regular schools, 
how Buch opportunity did the Career 
Education Projraa provide you for 
learning about occupations? 


113 


4.50 


4.29 


4.74 


32. in cooparison uUn regular schools, 
hcv =uch o?porta.Vcy did the C-rcc- 
Education Projraa provide you for 
general leaminr? 


113 


3.97 


3.76 


4.20 


33. In coRparison with past experiences 
in regular schools, how ootivated 
are you to learn in the Career Ed- 
ucation Frogran? 


112 


3.99 


3.69 


4.31 


Ave raxes 


112.7 


4.15 


3.91 


4.42 



MeaiK on ccalft froo 1-low to S-hijh 



a Strong satisfaction with the Career Education Program when compared with 
regular school programs • Learning about occupations was especially high 
(4.50). Again the ACE-Olney scores were more favorable in each case^ 

Omnibus Question / This section of the instrument consisted of a single 
item with sixteen subparts. The item asks for a rating on the importance of 
various types of learning, then a second rating on how successful the Career 
Education Program has been in effecting the learning. All ratings were given 
on a 1 = low to 5 = high scale. For rating importance the poles were "Not 
Important" to "Highly Important". For rating program effectiveness the poles 
were "Not Effective" to "Highly Effective". Results are reported in Table 
33. Students viewed everything to be fairly important. All mean scores 
were high, with an overall mean of 4.33 and a small range of 3.94 to 4.62. 
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T«bl« 33 
StudMt Opinion Survey Pr«ccsc 
OsDlbus Qucsdon 

leXov «r« lisccd sooe «rt4s of possible Isporcsnce for « scudeoc co lesm. 
PlcMs rsce csch lo terms of hov Isporcanc you feel Ic Is for « scudenc 
CO l««nit snd how veil you f«el the progra Is accoaplisbins ««ch. 



Subpsrts 


laporcance 


Effecdvtnaas 


Scudeots learn co: 

!• Fftrforo specific occ^)«cloa«l 
skills 


ToUl 
Mean 


ACE 
Mean 


ACE-Olney 
Mean 


Tocal 
Mesn 


ACE 
Mean 


ACE-Olney 
Mean 


4.18 


3.97 


4.32 


3.61 


3.11 


3.89 


b. le punctual and organize thalr 
timt 


4.31 


4.27 


4.33 


3.65 


3.20 


3.89 


c. Aasvse rcsponalbillcy for 
che&selvcs 


4. 52 


4.53 


4.52 


3.92 


3.56 


4.10 


i. Make decisions and follow 
chrough 


4.33 


4.38 


4.29 


3.82 


3.25 


4.10 


e. Cocaunlcsce vich ochers In s 
micure vay 


4.54 


4,63 


4.48 


•».71 


3.16 


3.98 


t. »e aware or sore career 
opporcunldes 


4.52 


4.52 


4.53 


4.37 


3.96 


4.51 


g. Work vich ochers 


4.35 


4.09 


4.52 


4.04 


3.68 


4.22 


i\. Evsluace chelr ovn work 


3.94 


4.00 


3.90 


3.73 


3.68 


3.77 


1. Perfors basic acadealc 
skills 


4.18 * 


4.36 


4.06 


3.74 


3.39 


3.98 


j. Think chrough and solve 
p rob lens 


4.42 


4.34 


4.47 


3.64 


3.16 


3.88 


k. Have a posldve acdcude 
ccward self 


*.51 


4.22 


4.69 


3.91 


3.52 


4.10 


1. Have a posldve acdcude 
coward work 


4.37 


4.22 


4.47 


3.71 


3.40 


3.87 


B. Have a posldve acdcude 
coward leamliu; 


4.62 


4.50 


4.69 


3.96 


3.32 


4.30 


0. Prepare for /trcher educadon 


4.24 


3.94 


4.42 


3. is 


3.3^ 


4.10 


0. Is:prove Incerpersonal and 
social skills 


4.05 


4.19 


3.96 


3.66 


3.29 


3.86 


p. Other (please specify) 


4.17 


4.25 


4.13 


4.00 


3.67 


4.17 




Averages | 6.33 


4.23 


4.36 


3.83 


3.41 


4.0/* 



Mesn on scale fros'^l^lov co 5-hlgh 

The lowest items were •"improve interpersonal and social skills" and ''evaluate 
their ownjork". ACE and ACE-Olney ratings did not differ markedly regarding 
importance. Regarding program elEf ectiveness the results were quite different. 
All mean ratings were lower than fpr importance, with the overall mean 3.83 
for effectiveness vs. 4,33. for importance. In rating effectiveness the ACE 
Group was again consistently less favorable than the ACE-Olney group in 
their evaluation. Although no score was especially low, only two scores 
exceeded 4.00: "work with others" and "be aware of more career opportunities". 



Other Items ♦ Seveyal other items were included in the instrument. 

Results are presented in Table 34. As can be seen from these responses most 

students a/te ' planning to complete high school. There did not seem to be a 

strong difference between the ACE and ACE-Olney Groups. Most students 
» 

did not have jobs during tfie school year* Many more ACE than ACE-Olney 
students reported employment, 'often accounting for a substantial number of 
hours per week. Very few students in either group reported any employment 
interfering with any other activities. 



TAble 34 
Student Opinion Survey Pretest 
Other Itecs 

Itea» : ' 

22. Do you plan to get a t^coodary school dlploca^ 



Itesponse 


X Tot.l 


Z ACE 


Z ACE-Olney 


L. Deflnictly No * 


8.3 


8.8 


7.7 


2. 


4.6 


5.3 


3.3 


5. 


22.9 


24.6 


21.2 


i. 


21.1 


22.8 


19.2 


>. Definitely Yet 


A3.1 


38.6 


48. 1 



3^. IXiring this school year have you worked outside of hoaer for aoney? 



Kttsponse 


X Total 


. : ACE 


2 ACE-Olney 


1 . No 


62.2 ' 


51.7 


73.6 


2. Yes, Isas than 10 hours 
a week 


J. I. . 


3.4 


— t 

11.3 


3. Yes, between 10 and 20 
hours a week 


u.; 


20.7 


' 7.5 


Yes, between 20 snd 30 
hours a week 


5.4 


5.2 


5.7 


>. Yes, Dore than 30 hours 
a'^veek 


10.8 


19.0 


1.9 



35. If you have an outside job, does It interfere with snythlng listed below? 



Response 


Z Total 


Z ACZ 


X ACE-Olney 


1. I don't have an outside 
lob 


65.9 


.56-3 


78.6 


2. My job doesn't Intwrfert 
with any other activi- 
ties 


''19.3 


26.1 


11.^ 


3. It Interferes with oy 
school work 


2.3 . 


2.2 


2.4 


4. It int<*r(eres with ny ' 
social Ufe 


6 « 


10.9 


2.4 


5. It interfrrc^ wLtb oy 
extracurricular activi- 
ties 


5.7 


6.5 


4.8 
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Sunanary . 

Summary of the four common instruments designed for utilization 
at the EBCE sites, only the Student Opinion Survey had been administered 
in time for analysis and inclusion in the present report. Results from the 
other instruments will be reported in Special Re^rts, Task Reports and the 
Final Report. In general, students rated the Career Education Program highly ^ 
with particular emphasis onjhe program's interest value, individualization 
(^ a^ opportunity for learning. The more structured nature of the RBS progr^ 
was reflected in students' lower ratings of ability to determine the amount 
of time spent ir activities. The fact that these ratings were not extremely 
^ low. and that other perceptions were very favorable, would suggest that this 
is not a serious concern. It may not even be a criticism, but rather a simple 
statement of fact. If should be pursued to conclusion just in case a problem 
is there. Students evidenced a high vocational attitude, especially regarding 
their anticipation of employment. Most students seemed to think that'gaining 
knowledge about careers and jobs was the most important reason for entering 
the Career Education Program. Students vere strongly favorable toward the 
Career Education Program, in comparison with traditional school programs. When 
presented with a series of learning objectives, students thought all of them 
were important. Infetpersonal and social skills and self evaluation were con- 
sidered-.the leaSt important. The ratings of program success in achieving the 
' objectives were ,riot as high as the rated importance of , the objectives. The 
difference was fairly consistent, but difficult to interpret because the Zero 
Point for importance and effectiveness may not be the same. For each objective 
, the ACE Group ratings of effectiveness were lower than the ratings accorded by 
the ACE-Olney group. This was consistent with the other results obtained-; thus 
it would seem that twelfth graders had a generally less favorable attitude toward 
the program than tenth and eleventh graders. Most students planned to get 
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a high school diploma. Only a minority were working during the school 
term, but some ACE Group students were wdrking a substantial number of 
hours per week. Very few students replied that their vocational activities 
interfered with any of their other activities. 

VIII. INTERIM SUMMATIVE EVALUATION RESULTS 

This section of the report has been reserved for specifically summative 
results. Of the summative documents listed above in the "Summative Evaluation 
Overview**, reports have been completed on the ^^a^agement and Support Systems 
Components. The present report will review the results of these summative 
documents. In addition, the results from the pretest-posttest administra- 
tion for seniors have recently become available* A preliminary analysis 

"A 

of these results will be presented. These results must all be regarded as 
tentative since the experimental year is still in progress. Extensive sum- 
mative data and conclusions will be presented in the final report after all 
the information has be^ collected and analyzed. 

Management Systems Components 

The area of management systems was defined Sls containing the following 
components: 

1. Policy Determination 

2. Institutional Relationships 

3. Program Administration 

4. Planning and Formative Evaluation 

5. Community Relationships 
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These components were reported upon to the extent allowed by the data 
available in Summative Report 1, "Report on Management Systems Components." 
Conclusions from this report will be discussed briefly below. Priorities 
stated for each area ar-e from the Operating Plans for FY 1974. 

Policy Determination , This function falls mainly to the Board of 
Directors of the Academy for Career Education. For most of this year the 
Board has been constituted of seventeen representatives including employers 
(11) educational agencies (4), community (3) and labor (2). The following 
priorities were established for the current year: 

1. Continue refinement^ and maintenance of an effective policy-making 
group, the Academy Board. 

2. Increase the policy involvement of individual employers. 

3. i^brease labor union representation on the Academy Board-. 

4. Increase student and parent participation on the Board. 

5. Systematize codification and dissemination of Academy policies. 



To date, each of these priorities except the last has occasioned successful 



developments within the Policy Determination Component. The desirability | 
of the last priority has come into question in terms of such sys tematization 
promoting institutional rigidity and creating a mountain of red tape. The 
issue is currently under discussion, and it is likely that a more flexible and 
informal way to attain clarity of police's will be sought. Progress in stu- 
dent and parent participation in Board activities has been somewhat slow due 
to the lack of organization in student and parent advisory groups. It is 
felt that student and parferi? Board members can be considered representative 
of their constituencies only if they are recommended by organized advisory 
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groups. Development in this area thus awaits the development /6f advisory 
groups. Progress in other areas has been according to plan. The Board 
has continued this year to be a committed, active, constructive and critical 
group. 

Institutional Relationships > The Career Education Program must maintain 
relationships with the Pennsylvania Department of Education, the Greater 
Philadelphia Chamber of Commerce and the Philadelphia School District* The 
success of these relationships determines the success with which the instruc- 
tional program may be conveyed to students. T>ie following priorities have 
been established for the cutrent year: 

1. Establish a cooperative relationship between the Academy and the 
Philadelphia School District. 

2. Maintain the private academic school license. 

3- Establish a cooperative re- la t lo nship between the Chamber of Commerce 
and the Philadelphia School District, 
llie Academy (and CEP) has been able to establish effective institutional 
relationships with each of the agencies necessary for conducting the program 
and preparing for replication. The third priority, which calls for promoting 
a relationship between two putside agencies, has not met with great success 
to data. The effort is still in process; thus any conclusions must be 
regarded as preliminary. Otherwise, this area has proceeded very well. The 
^ relationship with the State is substantially perfunctory, but does require 

a continuing effort. Relationships with the school district have required 
extensive effort and resources. This investment seems to be paying off in 
terms of district cooperation and the development of good working interrelations. 

Er|c 77 
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Program Administration , The RBS Career Education Program Director is 
responsible for the performance of ^the EBCE contract* His conduct of the 
program occurs within the framework of policy set both by the Academy Board 
of Directors and the' EBS Board of Directors. He serves simultaneously as 
the Academy's Executive Director and the Career Education Program Director. 
His position is responsible for supervision of all staff, program events and 
management decision-making. The following priorities have been established 
for the current year. 

1. Maintain a balance between centralization and decentralization^, 
i^e., centralization of certain functions such as submission of 
reports to NIE and decentralization of others, such as budget 
control within components and decision-making on the component 
level. 

2. Manage the involvement of the Chamber of Commerce and the Phila- 
delphia School District. 

3. Establish a system for tracking and projecting operational 
costs. 

4. Increase the quality and decrease the quantity of reporting to NIE. 

5. Reduce external demands on project resources. 

Each of these priorities has been approached and fairly well met. Centraliza- 
tion of products has been achieved with the Program Director serving as the 
monitoring and coordinating resource. Supervisory staff in Career Development, 
Career Guidance, Basic Skills, Replication, Evaluation and Administration have 
been accorded a high degree of autonomy in fiscal and other decision-making 
within their areas. Management of relations with the school district has 
been conducted. by the Public School Liaison Officer, a staff member, with 

r 
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Input from the Program Director* Management of relations with the Chamber 
of Commerce has been accomplished by the Career Development and Career 
Guidance Team Leaders with input from the Program Director, The latter 
relations have not been as satisfactory as the former, with some difficul- 
ties cited by all parties ihvolved, A new cost tracking system has been 
developed and instituted by the Program Director. This system includes 
seven major cost centers: Program Administration, Evaluatj^n, Replication, 
Career Developij^t, Career Guidance, Basic Skills and Supplementary Program. 
Within each arercost subcenters which focus on various areas of development 
and operations. After each cost has been attributed to a use by cost center, 

categorized substantively. This is done by assignation from a chart 
of ac^^ounts containing approximately fifty categories ranging from Books and 
Subscriptions to Salaries and Wages. This cost system seems to be efficient 
an'd inforipative. Concerning report quantity, as of this date last year^ a 
total of 964 pages of formal task report had been submitted to NIE in the 
form of 6 documents each averaging some 161 pages. To date this year, only 
534 pages of formal task report have been submitted to NIE. These were in 
the form of 16 individual reports averaging 33 pages each. This represents a 
quantity reduction of 45%» Evidence is thus available that the quantity" 
of reporting has^ in fact been significantly reduced this project year contpared 
with last project year. The reduction in bulk per se has made rtie reports 
more useful to project staff anti^ presumably to NIE and external reviewers. 
In addition, the reports have been of a generally higher quality. The 
increased quality and decreased quantity seem to have resulted" in reports 
which have been clearer, more consise and more useful. External demands 
which require resources are usually exerted on experimental and demon- 
stration projects. These can range from simple inquiries to extensive t 
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requests for information or servi^ces^ The' project management has been 
sensitive to these external demands and tried to allow inclusion only of 

y 

those directly relevant to parts of the task scope. 




Planning and Formative Evaluation . This component is concerned with 
developing evaluative information which is useful in program management and 
planning. Its activities are a joint function of the evaluation and 
administration staffs. The following priorities were established for the 
current year: 

1. Establish extrinsic standards for comparison of Academy students 
with '"traditional** high school students. 

2. Develop a basic information system aimed at the compilation of 
complete and consistent program data. 

3. Develop a system for cost analysis. 

4. Develop a reporting system focused on evaluation usability includ- 
ing assessment of actual uses. 

These priorities, 'except the last, have been pursued virtually to com- 
pletibn* Determining the use of evaluation data is a task still under 
development. This remains a priority for the second half of the year, the 
time when most of evaluation reports will be produced. In the other 
areas, extrinsic standards have been provided by the establishment of the 
student populations discussed above. These student groups will allow a 
comparative analysis of program effects. An extensive \iata system has been 
developed and implemented this year by the evaluation staff. It is under 
further development, but at present is productively functional. The cost 
tracking syst^em outlined above will enable fairly precise and detailed eval- 
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uative cost analyses. The first attempt at this is presented in the 
"Formative Evaluation Results" section below. 

Community Relations , This has not been a priority area in the program. ^ 
Little publicity has been sought in the past; parents have been the only 
group whose support and interest has been sought. New importance is added 
by the prospect of replication sites, and efforts are now targeted in this 
direction. This component is the responsibility of administrative and 

) 

replications staff. The following priorities have been established for 
the current year: 

1. Provide a better defi^n^ion and analysis of the constituent 
publics in order to effectively deliver replication specifications. 

2. Improve and expand the relationships with the Philadelphia School 
District, .Chamber of Commerce, participating employers, parents 
and students so that the program can continue in the future. 

The first priority is presently being pursued through proposed replications 
studies, the activities of the replications task foVce , the 'activities of the 
Replications Advisory Group and written products of, the replications staff. 
IThe area is still % developing one and conclusions would be premature. The 
second priority has been discussed above with regard to the Chamber of 
Commerce and School District. Parent meetings have been held again this year 
with improved attendance and interest. Student government is still 
struggling ' for establishment ;** interest seems to be confined. Employer 
involvement is still strong. Some problems .in coordination' and support have 
been suggested by the results of employer interviews, but full results are not 
yet available. Overall, the support of individuals and institutions involved 
in the program has been more than sufficient to enable its continuation. 

HI 
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Support Systems Components 

The area of support systems was defined as containing the followiog^ 
components: 

1. Staffing ^ 

2. Students 
3» Logistics 

4. Supplementary Program 
These components were reported upon to the extent allowed by the data 
available in Summative Report 2, "Report on Support Systems Components." 
Conclusions from this report will be discussed briefly below. Priorities 
stated for each area are from the Operating Plans for FY 1974 . 

Staffing . This component is concerned with the provision of staff 

/ 

adequate to perform the workscope of the Career Education Program. The 
selection and supervision of staff are the responsibility of the Program 
Director and the unit head in- each area. The professional staff presently 
numbers 32, distributed as follows: 

1. Program Administration -■ 3 

2. Evaluation - 4 

3. Development ' 10 

4. Operat ions - 12* 

5. Replication - 3 
TOTAL ~l2 — 

* 8 of these are Chamber of Commerce staff under subcontract to RBS. 
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The following priorities were established for the current year: 

1. Enhance articulation between development and operations. 

2. Enhance articulation across teams aixd components. ^ 

3. Respond to the demands of the workscope. 

4. Meet training needs. 

An undesirable polarization of the staff into operational and developmental 

contingents last year resulted in an effort this year to unify and int^rate.' 

A major structural change in this direction was the ^reorganization of sta.ff ^ 

into content-referenced units ( e.g.. Career Development, Basic Skills, etc.) 

rather than function-referenced units ( i.e. , development and operations). 

This new arrangement has resulted in observable^ improvement in staff inter- - 

relationships, but Xhe onset oi the Chamber of Commerce subcontract has introduced 

some problems. .'The Chamber staff is distinct from the program staff in terms 

of phys-icaL location and program function. Articulation and communication has 

become a problem in *some areas. It has been attempted to overcome these problems 

through weekly team meetings between program and Chamber staff, weekly program 

cabinet meetings which include the director of the Chamber staff, and individual 
t' ' • 

contact. 

Articulation across teams and components has teen greatly enhanced this year. 
The cabinet; which has been expanded to include representatives from each team,, 
meets weekly^ to discuss issues, development and plans^ 'Jhe task reports, which 
are more concise and readable, are distributed to all staff in a timely fashion. " 
There are weekly st^ff meetings of each teajp. There are periodic full-staff 
meetings, retreats ^nd other occasions -which allow dissemination of information 
and interactioj. These resources are available; the actual extent of their effect 
has not been a:scertaiped . The* demands ' of the' wpiikscope have been well met. 
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The task schedule is current; and the intentions of the operating plans 

have been realized. Meeting training needs has presented some problems. The 
diffifculty has been largely confined to the Chamber staff, which is the only 
newly staffed unit. The difficulty arises from the fact tliat last-minute 
contract approval did not allow sufficient time for either recruiting or 
training this staff. Thus, training has>-extended over the operational year. 
Training in other areas has been adequate. 



Students > This component is concerned with providing students to 
participate in the Academy program. The responsibility f or Irecruiting students 
has rested largely with the Administrative Head of the Academy, the Public 
School Liaison Officer, the Career Guidance Team Leader, and the^yCareer Devel- 
opment Team Leader; other project staff have also beefi involved in the effort. ^ 
At the end of The first quarter* of this^ academic year, 147 students were 
enrolled in the Academy program. The following priorities were' establish 

J,or the current year: ' . ~ " 

1. Obtain and retain a cross-section of students. 
• 2. Relate student Selection criteria to student perforp;iance . 
3.. Attract and^ interest a large number of, applicants. 
4. PromotjB student understanding of the program. 
Since the student populations have been extensively treated^ in sections 
above-, the .present discussion will be brief. Two other evaluation reports 
may be consulted for additonal information: Special Report 1, '^Student 
RecruitmenT and Selection," and Special Report 2[ "Student Cbaracter- 
/ istics". A fairly good cr6ss-;section oT students was , derived from 

last year's and this year*s recruiting, efforts . Scores on all measure'? px- 
^hibit wide ranges with the relatively low meayk^'^c^tStXi^t^istic of present^ urban 




^ 60 = 

populations* The 6% r^te of attrition over the course of the first quarter 
indicates good holding power, since irfitial drops are usually heavy. No 
past or presently measured student characteristics have been found to relate 
to student performance in the program.. This could mean that there are no 
predictors,,^ or that we aren't measuring them, or that we aren't detecting 
effects » Resolution of the questions involved should be possible after the 
posttest data have been cdllected anJv^analyzed. During recruitment, a total 
of 426 students expressed interest in the program. Of the^se, 361 actually 
applied for entrance to the program* These large numbers of students indi- . 
cate substantial interest in the Academy, especially when the short time 
available for publicity and recruitment is considered. The time frame also 
hindered the student orientation process. Level of program understanding was 
not measured. 

Logistics . This component is concerned with the activities necessary to 
support the instructional systems including: facilities, transportation, health/ 
safety, insurance, business management, instructional materials, supplies and 
equipment. These matters are principally the responsi-Bility of the Program 
Director and other staff with administrative responsibility. The following 
priorities were established for the current year: 

1. Maintain responsiveness to logistical problems* 

2. Maximize procedural efficiency. 

3i Maximize procedural ef fectiveivess* 
No major problems of hote have occurred in the handling of logistical issues. 
One area whlchS^gs^Jme subject of much complaint last year. Academy facilities, 
was ^greatly -improved this year by reallocating space. / ' ^ 
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Supplementary Program . The Supplementary Program is charged with the 
responsibility of providing twelfth grade rs^jsdtli courses, in addition to the 
core program, to meet credit requirements and pursue their individual 
interests • The Supplementary Program is managed by a team leader and opera- 
ted by a program coordinator. Since formal priorities and decision-making 
criteria, were not stated in^the operating plans, an^inj^ormal review will be 
presented here. This program unit has provided approxi^tely 30 different 
learning activities each quarter. The activities offered included physical 
education, health, science, psychology, music, journalism and driver training. 
The instructors included RBS staff, conuuercial agencies, employers and raem- 
bers of the community. The Supplementary Program remains a viable and attrac- 
tive facet of the Academy program. A wide variety of activities are offered, 
and the program is favorably rated by both students and instructors. The 
program continues to be highly individualized and capable of meeting required 
credit needs and ^Jie^nterests of students. It is planned to incorporate 
some elements Of this program into the core program next year. 

fretes ttest Data 

-' - ' - < 

Twelfth grade students were posttested near the end of February. That 
time was'selecte^ in order to equate for inter-test intervals among groups and 
to assure testing of the seniors planning to gr-aduate in March. The e^irly 
testing introduces a bias against experimental subjects since the period of 
instruction affecting gains was only 5 months (approximately September 20, 1973- 
February 20, 1974). The statistical importance of this bias is not yet 
resolved; the appropriateness of comparisons between this experimental group 
(city-wide) and the control groups (Olney-only) is^ still under inquiry. At 
any rate "ther data are herein presented for descriptive purposes, only. .Results 
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must be considered tentative since the analyses possible in the time 
available were' not complet;^^*^ scope. The results from the Comprehensive 
Tests of Basic Skills (CTBS) were selected for emphasis because they pur- 
port to represent crucial areas of cognitive development. Other analyses 
will be presented in later reports. 

Table 35 displays the CTBS results in grade equivalent form* Students 
who had scheduled Individualized Learning Center (ILC) sessions were analyzed 
separately from those who were not in the ILC. Thi^ was done in an attemi:)t 
to isolate ILC effects on basic skills development. For students with ILC 
activities gra|ie equivalent gains over the five month period ranged from .3 
of a year to 1 full year on the various subtests, with an average gain of .6 

of a year. Reading;^Comprehension and Arithmetic Applications were most favor- 

» 

ably affected; Arithmetic Computation and Concepts were ^east favorably 
affected. 

For students without ILC activities the results were quite different. 
No change or regression of scores waS^ usually' the case. Results Ranged 
from a loss* of 1.1 years to a gain of 1»3 years on the various subtests, with 
an average loss of .3 of a year. Reading Vocabulary was most negatively 
affected and Arithmetic Applicat ions was most positively affected. 

/ 

Table 36 displays the same results computed in scale scores. These 
scores were used ^or statistical analysis of the- reliability of observed gains. 
Statistical tests for correlated data pairs yielded the T values shown. The 
* indicates change scores which may be accepted as reliable at the 957. level 
of confidence* For the students with ILC activities all gains but Reading 
Vocabulary were significant. For the s^Uj^nts without ILC activities two* losses, 
Readlag VocabulSt|ry and Arithmetic Concepts, were found to ^e sigr^if icant . The 
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Table 35 



12th Grade CTBS Galn<?'ln Gradp Equivalent 



Students With ILC 




Reading 

VocabuJary 



Comprehension 



55 » 



55 



Pretest 
Mean 



9.2 



8.2 



Posttest 



9.7 



9.2 



Mean 
Gain 



+0.5 



+1,0 



Total 



Ati thme tic 

Computation 



55 



56 



J. 7 



5.1 



9.5 



8.4 



+0.8 



+0.3 



Concepts 



54 



8.0 



8.3 



+0.3 



Applications 



54 



6.7 



7.7 



+1.0 



Total 



■53 



8.1 



8.3 



+0.3 



Students Withoue' ILC 




Reading 

Vocabulary 



12 



Pretest 
Mpan 



13.6 



Posttest 

Mpan 



12.5 



Mean 
Gain 



-1.1 



Comprehension 



.Total 



Arithmetic' 
Computation 



Concepts 



12 



12 



10 



10 



13.6 



13.6 



11.9 



11.7 



13.6 



13.6 



11.3 



11.1 



0.0 



0.0 



-0.6 



-0.6 



Applications ' 



Total 



10 



10 



10.3 



12.2 



ll-. 6 



11.4 



j'1.3 



-0.8 
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12th Grade CTBS Gains in Scale Scores 



Students With ILC 



- ^.Statistic 
Subtest - - 


n 


Pretest 
Mean 


Posttest 
Mean — 


Mean T 2 
Gain Value 


Reading 

Vocabulary 


55' 


570.67 


581.02 


4-10.35 1.56 


Comprehension 


55 


549.56 


- 574.65 


+25.09 3.05'' 


Total 

■ a 


55 


557.53 


578.13 


+20.60 3.12'' 


Ari thmetic 

Computation 


56 


524.25 1 




+ 9.97 2.35'' 


Concepts 


-54 


e» { 

521.94 I 


5Q3.04 


+11.10 2.19=' 


Applications 


55 


503.19 


532.80 


+29.61 3.95" 


Total • 


5T 


-.514.02 


531.85 


+17.83 3.13^^ 



Students Without ILC . 



7=- :-&tatistic 

Subtest " ^ 
: ' ■ — * 


n 


Pretest 
Mean 


-PuHtt^st 
Mean 


Mean. I2 

Gain Valufe 


Reading 

Vocabulary 


12 


709.83 


678.67 ■ 


-31.60 2.55* 


Comprehens ion 


12 


' 675.50 . 


701.50 


+26.0 1.36 


Total 


12 


698.50 


701.17 


+ 2.67 0.18 


Arithmetic 

Computation 


10 


635.50 


618.80 


-16.70 0.87- - 


Concepts 


10 


636.80 


627.20 , 


9.60 2.28* 


Applications 


10 


605.00 


63^^.90 


+34.9 ^02 


i Total 


10 


633.30 


629.. 20 . 

i 4- 


- 4.10 0.'57 



1. Result of parCrtllpatio^Kin program for 5 instructionai- months, October' 
February. 

2. Statistical reliability of pretest-posttest difference, *pC<;.05 when T 
2.01 and df=50, p.<.05 when 1^2.23 and df=10. 



• 71 



other change scores were not statistically reliable at the^^tated level . 

From these results^/b^ may be concluded that the gains in basic skills 
observed among 12th grade ^tudents wete substantial. They exceeded national 
normative expectations and farther exceeded the average, .gains among urban 
populations. The results^ indicated, as last year, Reading resources are 
apparently stronger than Arithmetic. The lowest area last year, Arithmetic 
Applications, seems to have been remedied but the other Arithmetic areas 
need work. 

Applying the results of students who did not have ILC activities is more 
tenuous due to the small number af students involved. The preponderance 
of unfavorable gains does indicate that the ILC is fostering much development 
in basic skills. In addition to the small numbers of students, the possi- 
bilities of ceiling effect blur the interpretability of th'ese results. 
Further study at a more ^detailed level needs to be done of this phenomenon'. 
' ^ The slater availability of tenth and eleventh grade data may shed some light 

on the is^ue. 

The Career' Maturity Inventory (CMI) was also administered. Results * 
indicated no significant gains otv any of the stT^teSrs^ The CMI is currently 
under study, and later reports on the instruments validity will be issued • * 

The Assessment of Student Attitude Scale (ASA) was also administered. 
This instrument 'is still under 'development, and it was not possible in the 
time available to accomplish the necessary pi^etest-posttest conversions in time 
for the present report. - . ' ' ^ > 

^ • ■, / / ■ ■ . 
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Summary 

This section has presented st^btjLve^ results on the Management Systems 
Components, Support Systems Components and -pretest-posttest data on twelfth 
graders. Summative data at this time in the experimental year must be con- 
sidered tentative. Extensive analyses were not possible given the limited 
data available. Discussion of Management and Support systems was basfed 
largely on separate reports pi;eviously submitted for those areas. Pretest- 
posttest results represented preliminary data on twelfth graders only. In 
Management Systems^ the following positive trends were noted: * ^ 

1. continuing development of a strong and useful Academy Board of 
Directors . , - 

2. continuing licensure of the Academy 

3. extensive development of Relationship with the Philadelphia School ' 
\ District \ , 

4. less centralization cff project management 

5. developmerj/tf ^f a cost tracking system and approaches to cost analysi 

6. improvdme^^it in reporting system to NIE, 

7. establishmetrt >^ student con^arison groups 



r^^^ studen 



8. development of new information system 
The following' problem areas were also discussed: 

1. extent of policy codif icat'ion necessary 

2. " management of relations^ with the Chamber of Commerce 

3. devAopment of a reporting system focused on evaluation usability 
Each 'of the area§ listed in the operating plans as priorities was discussed. 
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In Support Systems the following ^^itive developments were noted: 

1. maintenance of adequate staffing f^i^ workscope 

2. provision of better resources for articulation across te^jns and ^ 
components 

3. recruitment of a large and diverse student body for this year 

4. maintenance of efficient logistical systems 

5. continuation of effective Supplementary Program 
The following problem areas were also^ encountered: 

1. articulation between program staff and Chamber of Con^rce staff 

2. needed improvement in training efforts 

More information on these issues may be found ''in Summative Reports 1 and 2. 

Analyses of the pretest-pos ttest res,ults. of the Comprehensive Tests . ^ 
of Basic Skills for tvelfth grade students showed significant gains in most 
areas of reading and ^arithmetic • The^ average gain was .6 of a year over a 
5 month instructional period. Students vho were not involved in Individuar^ 




ized Learning Center activities did. not fare well on thesd tests; in fact, 
an average loss was observed. The implications of these findings require 
further study. These results must be regarded as tentative since only a 
subgroup 9f students were tested and no control group data are available. 



^ it. INTERIM FORMATIVE EVALUATIOJ^ RESULTS. 

/ ^ 

' t . . ' 

The Formative , Evaluation section of the Interim Evaluation ^Report 
deals with analysis and evalua^tion^of ^aspects of *the Career Education Program 
which' are specific to the Reseafch for. Better Schools* implementation of such 
a program. Krlsfly, this^ report describes the program, the recruitment and 
selection of students and employers, the us4*of support systems and advisors, 
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the instructional services provided to students, and the cost of providi-cig 
those services, 

Description of Program 

The Research for Better Schools^ Career Education Program consists of 
four components which provide the learning experiences for^'^Aca^emy students 
participating, in the program. The four comjponents are:, the Careei^ Develop- 
ipent Unit; the Career Guidance Unit, the Basic Skills Unit, and the Supplemen- 
tary Program. A brief description of each'unit and its objectivPR follows: 

The Career Development Unit is designed to promote^ academic growth 
and aid. each student in developing a rational, reality-tested career plan 
which he coyld use to guide and shape career decisions in the future. 
There are two components of the Career Development Unit: Career Exploration 
an<^ Careet Specialization. Career Exploration encourages e^ch student to 
identify potential career interests, and Career Specialization allows each 

student to extend learning experiences Into a specifl\f area. 
, <> 

Career Exploration is a series of structured explorations of the world 

" / 

of work conducted in employer settings; these explorations allow the student 
to identify potential career interests. The objectives of Career Exploration 

i 

are to be able to classify careers into groups based upon various sets of 
criteria; to know the general rights and responsibilities of-wdrkers; to' ' \ 
understand the impact of the work experience on one ^ life and the need for 
making' a meaningful career choice; to be able to deal with the concepts, 
tools, and practices of business which ha>>e an impact on the individual as 
a worker consumer and citizen; to be able, to identify and demonstrate 



general wprk related academic proficiencies and general job acquisition and 
maintenance skills; and to be able to apply the decision-making process to 
the selection of one career area for in-depth analysis. 

To accomplish the objectives of Career Exploration, the Career Develop- 
ment Unit has organized sixteen groups or "clusters" of similar experience 
sites. From one to f inexperience sites have been used for each cluster. 
Clusters that are-ajrallable to the students are apparel, art, chemistry, 
communications, construction, education, finance, government, health, labor, 
manufac taring, marketing, personal services, research, transportation, and" 
utilities. 

Career Specialization encourages each student to extend his learning 
experiences into a specific area by participation "^in structured, individual- , 
ized, and career-specific experiences conducted in employer settings. There 
is an internship and residency aspect to the ^Career Specialization component. 
The objectives of the internship experience are to know the specific skills, 
abilities, alid academic proficiencies required in the career area;- to be 
able to develop a plan for obtaining the required skilly, abilities, and 

proficiencies; to identify and acquire ,the cntry-leVel employability skills 
required; to be able to identify both the lateral and vertical avenues for 
growth and development common to the career area; to be able to demonstrate 
beginning skill competency in at least one of the functional skills common to 
the career area; and to apply the decision-making process to the personal 
identification of a tentative life career* The objectives of the'^esidency - 
experience are to extend beginning skill competency in order to be able to 
demonstrate some specified leyel of achievement ^ the skills and proficiencies 
coiranbn to the tentatively selected occupation; t;o develop self-av^retiess and 
self-canfidence in the occupation; and to develop a personal plan and calendar 
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reference system for achieving identified career goals. The Careei^ Special- 

ization programs utilize many of the cluster experience sites for the intern- 
ships and residencies. 

The second of the four components of the Career Education Program is the 
Career Guidance Unit. The Career Guidance Unit provides Guidance Groups for 
-all students and one-to-one counselin;^ between a student and his' counselor- 
coordinator, li also supervises Life Skills ^Specializations. The objective o 
the Guidance Group is to identify and clarify individual Values, to promote 
interactive, planning., problemrsolving and decision-making skills. The 
objective of the one-to-one counseling is to_extend the student' s f elf-concept 
and self-direction through interaction and experientially oriented feedback, 
with the counselor-coordinator. The objective of the Life Skills Specializa- 
^tions is to define and relate Academy experiences in terms of academic achieve- 
ment, career, planning and training, and evolving life style through in-depth 
contribution to a social service community agency. 

"V. 

The third component of the Career .Education Program is the Basic Skills ' 
Unit. The overall objective of the Basic Skills Unit is to improve t,he 
proficiency of students in communication arts and mathematical skills. 
More specifically, the Basic Skills Unit has been designed to assure vthat 
each student attains at least .a ninth nmde >rof 1 clency-. as measured by 
standardized tests, in communication and mathematical skUls. An individual- 
ized learning format is used by the Basic Skills Unit to achieve these 
Objectives. Part of the curriculum material used is the Individualized Learn- 
ing for Adults (ILA) program which' is carefully sequenced and structured. The 
mathematics material is comprised of eight levels and five areas; there are 
282 behavioral objectives for th6 Tnathematics area of the lU materials. The 
communication arts material Is comprised of eleven levels and ten areas, and 
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^^"^A^^ behavioral objectives. The 3asic Skills Unit also offers students 
who have demonstrated <t ninth grade proficiency in general mathematics 
and communication arts the opportunity to p-articipate in other instructional 
programs which permit, the student to meet graduation requirements and expand 
personal interests. The nature of these additional instructional activities 
is either in the form of structured experiences or independent study. 

The fourth component of the C^eer Education Program is the Supplementary 
Program. This aspect^of the career education program is offered only to twelfth 
graders; the sending sehool, Olney High School, provides the supplementary 
program for 10th and 11th grade students. The objectives of the Supplementary 
Program are to provide activities which are necessary to meet state require- 
ments for graduation bub which are not offered elsewhere «in the program and 
to provide activities which reflect student interests and career plans. 
The objectives of the required activities are to develop activities to meet 
state requirements in physical education, to develop activities to meet state 
requirements in healthy and to'provide bptior^ for earning world cult/rtre ' 
credit. The objectives of the elective supplementary activities are to pro- 
vide a variety of learning activities which meet state graduation requirements 
for elecMve credit, to pro-vide lear^Kig activities in which career-related 
skills could be acquired, aifd to provide learning activities which meet 
student interests. To/realize the objectives of the Supplementary Unit, 11 
required activities In physical education, health, and world cultures and 18 
elective activities such as Spanish, music, typing, psychology, joumalisfla, 
driver training, and community development have been offered. The nature of the 
elective activities offered was determined from the results of a questionnaire 
polling students on the type of activities they desired. 
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The Career Education Program .provides a comprehensive program of explora- 
sj^ecializat ion in careers, which are coupled with guidance and 
ties to promote and facilitate a morq fully informed and . 
into an adult world. 



^. ^pport Systems: Staff and Facilities 

This part of the "Formative EvalXiation Results" will describe the staff 

\ 

\ 

and faci 1 i. t j^S_jise£^^n^^t^ of the RBS Career Education Pro- 

gram (see Summ^^tive Report 2, "Report on Support Systems Components", Feb- 
ruary 28, 1974)\ . 

There arV currently 32 full-time professional staff members working on 
the Research for fetter Schools* Experience-Based Career Education Program. 
These staff members are distributed among the program components in the 
following manner: 

Program Administration \ 3 

Evaluation : A 

Design and Developnent 13 

Replication . . » ' 3 

Car^^>-XLej^eIo-^w^ Unit 4 

Career Guidanc*e^a4t 3 

Basic ^ills Unit . . . . 2 

Supplementary Unit T^r .^^"t"'"T"^>^ . . 1 
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Operations j 12 

Career Guidance/Career / — ^ 

Development Units 8 

Basic Skills Unit . 2 

Supplementary Unit 1 

/ 

* Administration - *. 1 

A Table of Organization for Research for Better Schools/ Career Education 
Program is presented in Figure 2. ^ 

The FY 1974 Operating Plans stated the following decision-making criteria 
for judging the effectiveness of the staffj.ng component: 

1. Degree of articulation bjetween development and operation^.* 

2. Degree of articulation across teams and components. 

3. Ability of staffing pattern to respond to ihe demands of the task.' 

4. Ability to meet training needs. 

♦ 

Each of these criteria will be applied below to th^ operationalized staffing 
component: * | 

In FY 1973, the level of articulation and cotWunication between the opera- 

I 

tional and developmental staffs was judged unsatisfactory! This situation 

I 

was seen as the result of a de jure and de facto Reparation of th-^ two staffs 
in terms of physical proximity and lines of accoi|ntability . The operational 
staff was accountable to the Administrative Headjof the Academy, while the 
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develoomental staff was accountable to the Director of Planning. Currently, 
the operational and developmental staffs of each program unit are accountable 
to that! unites team leader. This new organizational scheme has led to an 
acknowledged improvement in all of the four units. Articulation and communica- 
tion remains a problem, however, in Career Guidance and Career ^Development . 
This problem Is seen as a function of the complex relationships between RBS and 
the Chambelr of Commerce. These relationships involve the RBS Program Director, 

V ^ 

the Chambet Project Director, the Chamber President, the Career Guidance and 
Career Development Team Leaders, 'and the staffs of both organizations. 
Establishing efficient working relationships has been a continuing task. 

Articulation and communication ^across teams and components are promoted 
by the following: 

1. weekly team meetings in' which the operational and developmental 
staffs of each unit meet to discuss problems, perceptions, etc. 
ha.v^ing to do with their Unit; 

2. weekly cabinet meetings in which the RHS Career Educat ion .Program 
Director meets with key project members to discuss the ongoing 
developments in the program; 

3. full staff meetings to apprise staff members of new ^developments in 
the program which might affect them directly; 

A. task reports which are distributed to the entire staff by the RBS 

Career Education Program Director; and ] 
5. occasional retreats intended to-educate staff about. new project 
. directions and to make policy decisions in concert with appropriate 
members of cooperatfing agencies. 
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It is believed that there is substantive and substantial articulation across 

^ teams and components as a result of the mechanisms which have been established. 

I 

.Nineteen tasks were scheduled for completion by th^ Career Education 
Program staff by February 28, 1974. Another eighteen t|sks were considered 
but eliminated from the scope Vof the project by the mutulal consent of t^e 
National Institute of Education and the Director of the kesearch for Better 
Schools Career Education Program. Of the nineteen tasks scheduled for 
completion, eighteen were completed in the time frame set by the operating plan 
and one was delayed. It is evident in view of these data that the current 
staffu^ pattern is well able to respond to the demands of the tasks stipulated 
by the contract with the National Institute of Education. 

The major problem concerning the training needs of the program seemed to 
be centered around the lack of available training^time prior to the commence- 
ment of FY 74 operations. The fact tHat the contract with the National 
Instj^tute of Education was not finalized until August 31, 1973 rendered train- 
ing programs for new s^ta^ less than adequate. This , coupled with the com- 
plexity of the Counselor-Coordinator role, resulted in an atmosphere of uncer- 
tainty in t^^^^Career Guidance and Career Development Units. Training needs 

m the other program components have been met satisfactorily. 
» 

The facilities used in the present program embrace four major site§. 
First, the Academy site at. Broad and Locust Streets serves as a central 
instructional location and provides^ o ff ice space for the instructional staff 
Ul^ie Basic, Skills Unit. Second, the RBS offices at 17th and Market Streets 

house some^^ the project staff and provide some space for instructional 

/ 

activities-'; Third, employer locations throughout the city are used for 
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most of the in^>tructional progranr. Fourth, the Counselor-Coordinators 
are provided office space at the center city offices pf the Greater 
Phlladclpliia Chamber of Commerce. 

The Academy facility is the central instructional site. The site 
selection was constrained by administrative requirements ( e.g, , reasonable 
proximity to employer locations and RBS offices), legal requirements ( e.'g. , 
school licensing and building codes), and instructional requirements ( e>g. , 
sufficient space to conduct the program). The major factors tended to involve 
cost, locati^, and codes. The site eventually selected is housed on the 
fifth floor and the mezzanine floor of a downtown office building.- The 
fifth floor is used for classroom space, secretarial space, the project's 
student records office and supplementary program staff office space. 
The mezzanine floor contains the Administrative suite, office space 

r 

for the operational staff of the Basic Skills Unit, a computer room, the 
Individualized learning Center, classrooms for the Career Guidance and Basic 
Skills Units and a student lounge area. Figure 3 shows the floor plans of 
the fifth floor and mezzanine, which comprise the Academy facility. 

The utilization of floor space presented in Figure 3 represents a basic 
restructuring, from last year, of the physical layout of the Academy. The 
Administrative offices have been moved from the fifth floor to the mezzanine 
floor, and the Individualized Learning Center (ILC) has been moved to a" new 
location. The move of the Administration offices has resulted in an increased 
level of rontact between the Administration and student body of the Academy. 
The move of the ILC has greatly relieved the noise and space problems encountered 

in FY 1973. An overall offect of the reallocation of space ha^ hpen a more 

f 

^efficiently and smoothly run operation. ' 



84 



, ^Figure 3 
ACABEIIY'FOR CAx£ER cDUCATION 
1405 Locust Street 
He22anin£ Floor 
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Employer Recruitment\and Selection 

The first step 4n the recruitment and selection of employers for parti- 
cipation in the Career Educatibn Program is the identification of a cluster 
system which organiEes the economic sector into manageable educational units. 
Every clustering system is based on dimensions perceived as most relevant to 
the overall objectives of the Career Education Program and the economic 
community in which the program operates. The clustering system provides a 
framework for the shaping of instructional .objectives , the developing 
of instructional experiences, and the management and supervision of the 
Career Exploration Program within the Career Development Unit. Sixteen 
clusters have been i'Sentified as being relevant to the RBS Career Education 
Prografu and the Greater Philadelphia economic community.' They were listed 
ear^lier in this section of the report. This is an addition of four clusters, 
^ or an increase of 33 percent to the program this year* 

Contact was made with the Greater Philadelphia Chamber erf Commerce 
during FY 1973. The Greater Philadelphia Chamber of Commerce was an 
established institution in the economic sector which had access to potential 
participating employers and which could provide and- promote contacts with 
employers which, in turn, could lead to the development arid implementation 
of Career Exploration and Career Specialization Programs, The Chamber of 
Commerce was presented with an overview of the goals and objectivj^s of 
Employer-Based Career Education. Further discussions were held which led to 
a strategy that divided responsibility between the Chamber and program staff 
for contacting, recruiting, and developing programs with various employers. 




The Chamber has had Che major responsibility for contacting e^mployers, 
and serving as liaison . between the economic sector* and the Academy for 
Career Education* 

Procedures which have been developed and systematized for the identi- 
fication and recruitment of employers fall into three phases. Phase One con- 
sists of the identification and recruitment of potentially involved 
employers (for more information see Task lOAl /T"qrniulaJ;£__E£0eesSesr>roce- 
dures and Materials for Identification and,-Reirt3aitment of Employer-Parti- 
cipants, and for the MaJjit^mfi^^ Continuing Refinement of their 
Invo 1 venent-'-i^i"'"''^ 

Procedures used in Phase One^are: 

1. Based oh the cluster system utilized, types of employers needed to 
..^^^^^^^^ implement the program are identified by the Career Education Program 
an^^-thase^^nee^ are relayed to the Greater Philadelphia Chamber 

o>^^OTnme rc e • /^^^ — "'""""""'"^^^^ 

2» The Chamber of Commefrce then contacts arT-o-i^icial , usually the chief 
executive officer, bf a company or business thaV fits^^^^o the 
cluster and briefly explains the purposes of the program and^^ites 
the company to participate. If the employer expresses interest, the^ 
Chamber informs him that^a Counselor-Coordinatox will contact a 
designated official of the company to describe the implementation of 
the program in detail. 

3* The Chamber then gives the relevant information to the Counselor- 
Coordinator who in turn arranges for a meeting and then provides 

t. 

information regarding Employer-Based Career Education, the Academy 
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>for Caree^-vMucation, the student population, aspects of the coop- 
erative arrang^nTwiti^^ and the objectives of 
Career Exploration and Career Specialisation. The Counsg^Lor-Coor- 
-dljia tor then asks the employer if he is in^fe^sted in being involved 
in th^ program. If the answer is yes, the emplo^^is ^sked to 
lesignate an Employer Coordinator to be responsible^^the develop- 
metit of a program. The Counselor-Coordirtat'^Hhen outlines kind 
of involvement that is needed, and asks the employer for any litet^^- 
ture that specif ideally describes the operations in his business or 
organization. .^^r — — 
f*hase Two of the identification and recruitment process is the securing 
of a commitment to develop a program in Career Exploration or Career 
Specialization. This is essentially the responsibility of the Counselor- 
Coordinator. The procedures used for developing a program are as follows: 

1. The Counselor-Coordinator arranges for a presentation with the 
Employer-Coordinator. This consists ^f how Career Exploration is - 
operati^V^lized, the function o f *the Academy for Career Education 
an\^ the interrelationship of its components, the goal of career 
education and the rationale for the cluster system that is being 
used, a description of the student population and its size, the 
number of days desired for Career Exploration, the specific progtam 
goals and how these ha^e been operationalized in the past^ 

2. The Counselor-Coordinator answers questions or concerns that are 
raised by the Employer Coordinator 

3. The Gounselor-Coo)?^<Jinator secures a commitment to begin a- series of 
planning meetings with the Employer CooTsidinator to develop a program. 



\ 
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Phase Three of the process of recruiting and selecting employers is 
the operationalization of the Employer Program Development Plan, This is 
primarily the responsibility, of the Counselor-Coordinator who, in cooperation 
with the Employer Coordinator, is to develop a program. 

The first step in the operationalization of the Employer Program Develop- 
ment Plan is the conducting of a site analysis. The Counselor-Coordinator • 
is responsible for conducting this analysis although full participation by 
the Employer Coordinator is encouraged. When completed, the site analysis 
guides the selection and collection of information and provides a data format 
that make^^^di^is information readily available. Information about the employer's 
organizatiori^^ah^operat ion is gathered by direct^^jip&l'views during which the 
^ empl-Qyex\is askedcb describe the major activities or functions that take 

place in his business or industry, by the securing of * catalogues, flow charts, 
annual "reports , and other organizational material already prepared by the 
employer, and by securing pamphl^s, books and job descriptions prepared by 
governmental departments and agencie^v.,^^ After this lnf(yrmation has been 
reviewed, the Counselor-Coordinator categorizes the employer site in terms of 
th^ clustering system and describes the major work activities and functions 
^\^^r^onned by the employer; these, in turn, are coded and their relationship to 
academic skill is identified, Oc^:upational data that are required for these 
activities are then listed; these occupational data include the job classifica- 
tiofis associated wi.th each activity and function performed at the employer 
site, the skills associated with eabh job classification, and theyacad^mic 
and vocational prerequisites, aptitudek, interests, physical demands and 
environmantal conditions related to the activities , and functions. 
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Once the site analysis has been completed, the Employer Coordinator 
and the Counselor-Coordinator begin to develop the employer program activity 
cycle. The four major elements^of the activity cycle arc the employer 
instructional objectives, the instructional activities, management systems 
and an evaluation system. The instructional objectives specify the intent 
flof the instruction in the particular employer program. The instructional 
activities are the means by which the instructional objectives are to be 
realized at the particular employer site; the site analysis'^ p^Q^S[|s 
basis for identification of these means. The management system spec 
sequence of events^ time and location of activities, supervising res 
and credit allocation. The evaluation system specifies the means of 
mining whether the program is meeting its objectives, a^ well as tim'&s 
formats of formal feedback to employers. 

\ The site analysis system described above was used to recruit the em 
who participated in the Career Education Program. Employers and Counselor- 
Coordinators both reported that It was extremely time consuming and, ^that 
it yielded a great' quantity of unused information. The s^te analysis "^pro- 
cedure accordingly has been revised; a new focused in-depth interview .format 
is currently being field tested. The new format uses a series of eighteen 
questions to determine aspects of an employelr site that a^re conmion ' ( eyR> , 
clerical activity at^ost sites, people ^ ope rating machi^^^s atTa maaufac-- 
tarings site, aj/d people gelling things at a department stttr^ characteristic 
( e>g. , surgeons for a hospital or teac?i^i;;s for a school), and\utiique ^.g> ^ 



glassb!i,c5wing at a specific site). 'Physical equipment and facilities^^ 
pr^/f^sses and systems, and genera'^^orking conditions associated' wi^th jobs in 
each of these categories are identx1^ied."^>^4back loops . have"'T>^eri built^ 
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into this systenj so* the Employer Coordinator can delete responses he feels 
are inappropriate, add* responses he feels have been omitted, and compare his 
specific site to other sites of its type. 

A total of 53 employers participated in the Cail?er Kducation Program in 
FY 1973. Only one employer dropped out of the program dur-ing the course of 
the year; an additional thirteen declined to participate in FY 1974. Of the 
fourjteen employers who declined to participate in FY 1974, six were not sought 
<ir continuation because their input was not considered adequate, . three had 



internal reorganization problems, two ^^giPt they had completed their conunitnient 
two felt that the students lacked interest and motivation , and two needed 
additional resources to continue. Thus, more than two- thirds of the employers 

participating in the Career Education Program last year have demonstrated a 

It 

continuing interest and commitment to the program. 

A total of 31 additional employers haVe been recruited and selected for 
participation in the Career Education Program for FY 1974. kidded t(/ the 39 
employers from FY 1973 who continue to participate, this makes a total of 70 
employers who have participated in the current year's Career Education Program 
The areas of the Career Education Program in which employers have .participated 
are listed below: 

NEW NEW CUMMULATIVE , 

FY 1973 FY 1974 TOTAL 

Career Exploration 31 25 56 

Career Exploration and; 

Career Specialization 8 — 8 

Career' Specialization 7 — 7 

Career Specialization and > 

Life Skills Specialization — 3 3 

Life Skills Specialization "7. 3 10 

TOTAL ■ / 53 31 84 
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. Of the emp]oyers pfirtlclpatlnp. In the Career Education Program this 
year, nine have discontinued their involvement in the program* Of these 
nine, two were discontinued by the program because their input was not con- 
sidered adequate, two reported they had insuffient manpower to provide a 
program, two stated that they were too busy for the number of students involved 
one cited insuficient finances to continue, one saw little value to the pro- 
gram, and one was involved in a major building campaign. This leaves 61, or- 
87 percent, of the employers recruited for participation in the program, for 
this year still commited to the Career Education Program. These 61 employers 
represent sixteen cluster experience areas. Career Specialization sites, ,,and 
Life Skills Specialization sites. The new employers selected continue to 
represent both profit and nonTprofit organizations. This diverse sampling 
of the economic sector continues to be consistent with the goals atid objectives 
of the Career Education Program. The composition of the group of employers 
in FY 74 represents an increase in both the scope and depth of the experience 

available to the' students enrolled in the Career Education Program. 

✓ 

Use of Advisors and Policy Making ^ 

Advisory groups for educational programs may be drawn from school district 
personnel, the business and industrial community, local community groups, 
parents, students and agencies. Representatives from each of these groups 
are used in an advisory capacity for the^Career Education Program. Selected 
representatives have been elevated to the status of active program policy 
determination by nomination and election to the Board of Directors of the 
Academy* for Career Education. 
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The policy iriaking function for the RBS Career Education Program is vested 
in the Board of Directors of the Academy for Career Education. Presently, 
the Board of Directors consists of twenty members: eleven employer represen- 
tatives, four education representatives, three community representatives, and 
two labor representatives. In their capacity as Board members, these indivi- 
duals do not officially represent the organizations with which they are 
affiliated. Eight of the eleven employer representatives are from companies 
that are participating in the instructional program. Of the four education 
representative^, two are from institutions of higher learning, one is from the 
Philadelphia School District, and one is a former teacher and teacher union 
leader. Two of the community representatives are from citizen committees on 
education, and the third is from a community-based educational organization.* 
The union representatives ^re from the garment workers and carpenters unions. 
In short, many traditional advisory groups have been e.levated to the position 
of active policy makers arid supervisors of the program. The Board meets at 
least once a month to consider and review the Career Education Program. 

The Board has further encouraged the formation of advisory groups by 
voting to grant Board membership to parent and student groups when these groups 
are sufficiently organized to elect representation. Groups which afe 
represented on the Board are also encouraged to act in advisory capacities. 
Employers not directly represented on the Board ar'e encouraged to -advise 

* 

the program director^ board members, or program staff as to courses of 
action th^y^^ffiight want followed. . ^ 

A stronger relationship between the Career Education Program and the 
Greater Philadelphia Chamber of Commerce has been promoted by several actions. 



operational aspects of the program.- The President of tjhe Chamber has agreed ' ' 
to attend, and "has" attended, all Academy Board meeting<j i'n an advisory capacity. 
The Academy Board also decided that the indijvidual 'participating employers are 
to have an effective voice in the selection of students for FY 1974. 

Utilization of the Philadelphia School District as an advisory group has 

been ipcreased iji the past year. In addition to. membership on the Academy 

■ ^ ° i 

Board, continuous contact is maintained with several personnel of' the district 

" ' 1 

to promote a close relationship between RES and the- Philadelphia School Distrlctj- 
The Philadelphia School District and RBS has facilitated ' this relationship 
by approving the Director of Alternative School Programs, for the Philadelphia 

School District is liaison between the two groups. * 

! • ' 

Efforts haxfe also been' made to estabjish and extend the cooperative 



relationship with( Olney High School. Oln^^ 
_to Its student ro^s, opened its faciliti^i; 



High School r^^s granted RBS access 
during the shimmer mo.nths for the 



recruitment and selection of students, macj/e available the academic records of 
those students, make attempts to reschediil 



le students into Academy activities, 
agreed to grant credit for Academy coUrsLs, and agreed to award a diploma to 
Olney students who successfully complete] the Academy program. The administration 
of Olney also played k significant role in the recruitment and selection of 
^students for the curreht school term. Among the suggestions adopted by the 
Career Education PrograAi were the preference given to 11th grade students over 
10th grade students and the preference given "to IGth^radq students who had 
attended Olney the year before. The Olney administration has also been consulted, 
for advice regarding the formulation of the Career Education Program for next 



an 
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The Research for Better Schools Board of Directors ac^s in a supervisory 
d advisory capcicity to th^e Career Education Program. / Through the RBS 

)cerutlve Director reports to this group and receives their recommendat ioijjs 

/ 

r tlie Career Education E^rogram. 

The National Institute o^f Education, the funding agency for the Career 

ikiucation Program, has been a valuable resource for consultation, advice, > 

d program direction^ Reporting ^and correapondence are forwarded to NIE on 

1 regular basis ;'>infonnal and formal contacts have been most useful in 

refining the Career E(Jucation Program. NIE's site visit in February 1974 

Droved useful^ as have the special meetings in Oakland, California in 

January 1974 and in New Orleans in February 1974. Special evaluation meetings 

conducted by NIE in Portland, Oregon in December 19173 and in San Francisco 

in March 1974 have provided a valuable resource for collective approaches 

to problems and issues. \ 

A special Academy Board Retreat was ,held in January 197^ this retreat 

* 

^ I ' 

was attended by representatives of most of the groups listed ^above . In^ all, 
35 persons represen^ij:ig the Academy Board, tbje -i^S Board, RB^^staff, NIE 
;taff, the Greater Philadelphia Chamber of Cojm^igrSi^-the^Phil'adel^^ School 



district, and participating employers attended this-- rat re a^2 
cfertn of this retreat was the current status and future direction of 




Career Education Program. Among the general sessions and woi:fe5fiops conducted 

ad' the retreat were the nature and role of the institutions involved in the 

1 , '> 
program, funding potential for the future, specific aspects of the Career 

Edjication Program, and the location of the Academy. 

Participants in the retreat completed questionnaires at the conclusion of 

eadh session. Table 37 presents the results of this informal evaluation. 
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Table 37 
Acadeoy Board Kccrcac 
EvaluAdon Questions 
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9. Workshop #2 

Chaubcr/Eeiploycr 
rcrnprcttvi' 



10. Workshop #2 

School Dlsxrlcc 

KwPf^cjLjyj:, -i<r^OO X' * 'IL% , , ^z?^ 

Ql - U.'ii this Crncral -Session or Work'.hop worthwhile ac trndliif/* 
Q2 "Did It acconpllsh Ics ohjr>ctivd? 

I.1VO a cloarot undcrodo^In;; of the topics and Issues covered? 




• Do yoii agree vlth the Mjor conclusions or decisions reached' 



Scale • 



Defln 
So 

I 1 



Curly 



Definitely 
Yes ' 



Each session is listed aovm the left-hand side of the table, while the questions 
are listed across the top. Question contfent is included below the tablev - The 
scale used for all questions ranged from L = Definitely NO to 5 ? Definitely YES. 
In all cases, higher numbers equal more favorable responses. The mean responses 
for each session by question are presented in the table. 
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The overall, opinion of the retceat was very favorable; most responses 
wBre4nthe 4 to 5 range. Individual session' ratings on "worthwhileness" 
ranged f^ffi^. 71 to 5.00 with an overall mean of. 4.45. Individual session S 

fings on "accomplishing objectives" ranged from 2.50 to S.OO'with an over- 
all mean of 4.13. Individual session ratings on "clearer "understanding" \ 
^\anged from 3»40 to 4.80 with an overall mean of 4.19^. -Individual session 
ratings on "conclusions or decisions" ranged from 3.50 to 4.67 with an over- 
all mean of 4.36. While the ovei^ll ratings for each question were high^ par- 
ticipants' favorable perceptions of the retreat's "worthwhileness" and 
--**4greement with conclusions" exceeded their estimation of "accomplishment of 
objectives" and "clarification of understanding." ^ '"^ 

One source of advice that should not be overlooked in this report is 
the other Experience* Btas^d Career Education Programs. The cooperative efforts 
by all EBCE programs have resulted in the Common Instruments 'which have been 
described in this report. Eff'orts to address difficulties and find solutions 
to various problems have occurred and resulted iii a^mutuaT refinement 
of all progran^ The Northwest Regional Educational Laboratory is analyzing 
" the-Career Maturity Inventory and will forward results on its validity to 
the other EBCE programs. .Research for Better Schools is analyzing the Common 
Instruments; each project staff is participating In their development. This 
cooperative mutual advisoiy* relationship between the programs has resulted in 
a better understanding of the. different approaches used and a sharing of 
helpful 'ideas. ^ , . $> ^ 

Student Recruitment and Selection 

Student recruitment and selection in 1973 consisted" of the selection and 
enrolling of 10th and 11th grade students who would participate in the core 
aspects of the Career Education Program (Career Development, Career Guidance, 
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and Basic Skills) and receive the supplementary aspects of the program from 
their sending school. The' noal of the selection and recruitment process was 

I 

the enrollment of 85 students in 10th or 11th grade for the Fall 1973 Academy for 
Career Education Program. This process has been described more fully in 
Special Evaluation Report 1, "The Student Recruitment and^Selection Process," 

• January 30, 1974. . ^ ' ' % . 

} ' ' ' 

The student recruitment process consisted of three^phases. The initial * 

phase was begun July 27, 1973* with the notification of 948 10th grade and 413 
Llth grade Olney High School , students of the opening of applications foi: the 
Academy for Career Education Program. Students wlho were Interested in the 
Academy program were invited to attend general information sessions held 
August 6 and 7, 1973. These sessiortr^re d^H^ned-<oexplain the Academy 
program's goals and objectives; the sessions were conducted by the Administra- 
tive Head of the Academy, the Public School Liaison Officer, the Career 
Guidance Team ^^eader, the Chamber of Commerce Project Directw^two other RBS. 
staff members and .one employer representativ^e. A total of 426 studett^ attended 
these general information sessions; the students were divided into^groGp^^^ 
to 12 and 30 to 40 minute* sessions were held with each group. 

^ The general information sessions led to applications being coir^^Vtect-^nd 
__r^iLumed by 361 students. Of these, 85 were accepted for the program on the _ 
basis of 10th or ilth grade status, intact records, completed applications 
which induced parental permission for participation, acceptable mental and phy- 
sical capability, and acceptable attendance, grades, and conduct. 

Of the 85 students accepted for the program, only 40 indicated that they 
would enroll. Thfe, fcess than desired enrollment necessitated a second phase 
of recruitment. Tpis phase consisted of the review. of the applications of 



the*276 s 



udents who did not meet the admission criteria of the first phase. 



By August 27, 1973/54 ad'aittdhytwii..;^^ were seJLetTted fx)v the Academy 

Program- These students were ctn^sidered marginal and rejected in the first phase 
of student recruitment. Preference wa^ gi^en to_White students, particularly 
White mal^bs, in order to attain a racial balance in the enrolled population.' 
i Seven of these students irldicated that they would en^i-Qll. In this second / 
phase, 200 students wete ag^in considered ineligible for the Career Education 
Pro gram y , . ' • / 

In the first two phases of student recruitment, the following reason^ 
were given for declining to participate: ; 
1. extracurricular activities at Olney High School,. 
/ 2* fear of losing contact with friends '\ 
3* close proximity. to after-school jobs 

4. misperception of the Academy as a vocational trainingfcent^r. 
' Since only 47 students had been enrolled in the Eirst two phases 'of 
student recruitment, ^C:;EBarS <^hase was conducted by the Public School Liaison 
officer* Students wh o had n^^ ^allv applied, but had expressed interest 
in the Acade my program^^wigtn^ interviewed and recruitj^ at Olney 

High 

recruitM werl"""ai^eiJLted and 
A^to\al of 69 s 




By August 29, 1312Lx.n studeilfes who had been individuWly ' 



n tho^cademy program. 





ogram at \the end of the thf^ 

sex, race, previous att 



Academy for Career Education ^ 

■ ^, ■ ■■ 

tment* Characteristics^ of 



grades o.f all, \ 



applicants were analyzed to determine if^^. dif ferenSS^ existed between those 
^tudents who were accepted for the program and decided to, enroll, ehos^ st?u^ents 
who ;J&re accepted for the program and deoJLined to enroll, and those students! > 
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who were considered ineligible for the program. Information was complete only 

for the grade level and sex of the applicants; data were incomplete for race, 

previous attendance, and previous grade average of the applicants; this was 

especially the case for those students who were considered ineligible for the 

program. The, quantity 6f incomplete data restricted most comparisons to those 

between the students who enrolled and the students who were accepted for the 

program but declined to enroll. 

Analysis of the grade level data indicated that 38 percent of the students 

accepted for the program were 10th grade students and 63 perceatj were lltli grade 

students. Of the students who were considered ineligible for the^pr.ogram^^ 

85 percent were 10th grade students and 15 percent were 11th grade students* 

* 

Table 38 presents the descriptive data for grade level of the recruited student 
sample. 



RfcrulcetJ Student Sanplc: Crodo Level 



Student 

Ch:kracterl«tlc^ 


Accepted 


Enrolled 


Accepted/ 
^^op^ed 


Rejected 


Total 


Grade Level 

Tenth Grade 

Eleventh 
. Grade 


# 


X 


# 




9 


X 


- # 


X 






102 


37 
63 


35 ' 


51 


25 
67 


27 
73 


169 
31 


^5 
15 


V 228 
/ 

133 


' 6^ 
37 


• Total 

Mean 


161 


100 


69 < 


' 100 


92 


100 


200 


100 


361 


100 


80.50 


3A.50 ^ ^ 


A6.00 


100^0 


180. 50 



100 



Chi Square analysis of the data revealed that significantly more 11th grade 
students were accepted for the program than was expected and that si'gnifi- 
cantly /fewer Hth grade students were rejected than was to be expected, Th6 

\ 

Chi Square analysis also indicated that a significantly greater number of 11th 
grade students declined to enroll in the Academy program after they were 
accepted. The .CHi Square analysis is presented in Table 39* These 
statistics reflect the intended preference given to 11th grade^ students as 
well as the suggestion of the Olney' administration that preference be given 
to 10th grade students who had attended Olriey High School the previous year. 



Table 39 

Chi Square 
Itecrultcd Student Sac^tle Cradc level 



Disposition 


T<nth Cradc' 


Eleventh Grade 




of 






Chi 


Students 


<5 ' chi-square 


chl square 


Square 


Enrolled 


43,61 34 


25.39 35 




0-69 


2,1176 


3.6373 


5.7549 


Accepted/ 
Dropped 

n - 92 


58.14 25 
18.1899 


33.86 67 
32.4353 


51.32^2 


Rejected " 


126.40 169 


73.60 31 




0 - 200 


14.3572 


24.6570 


39.0142 


Total 
N - 361 


p " .632 


p - .368 


96.0943 



df • a, Ci- .05, critical value - 5.991 
df * 1, s> .05, critical value - 3.841 
* expected frequency 



f^ ■ observed frequency 



Chi Square • £ (f^-fp)^ . 




-Information relating to , the sex of applicants was analyzed; this 
mation is'ix^esented in Table 40. Forty-eight (48) percent of the students 
accepted for the prci^am, were male and 52 percent were female • Of the stu- 
dents who actually enrolled iji the program 59 percent were male and 41 percent 
were female. For the students who^ere a^^^ed^ but declined to enroll in 
the program this ratio was reversed, 40 percent wer^ male and 60 percent were' 
female. Thirty (30) percent of the students who were cor?H^e^d ineligible 
for the program were male and 70 percent/ were female. 



. Table 40 
Recruited SCudcitC Saitpic: Sex 




Student 

Characteriatlc 


Accepted 




Enrol led 


Accepted/ 
Dropped 


Rejected 


Total 




1 ' 


X 










X 




Z 


1 


X 


Kale 


^ 


/«8 


41 






37 


40 


62 




31 


lAO 


39 




83' 


52 


28 






55 


60 


138 




69 


221 


61 


Not Available 


0 


0 


0 




0 




0 


0 




0 


0 


0 


Total .43^ 


i6r 


100 


69 


100 


92 \ 


100\ 


200 


100 


361 


100 



Chi Square analysis of the data indicated , there was no difference between 
the numbers of males and females accepted for the program. The Chi Square 
afnalysis is presented In Table 41. However, when expected frequencies for the 
Chi Square statistic were based on the total number of applicants, there werex - 

'A 

significant differences found; this information is presented in Table .^42. ,"1 
significantly greater number of males enrolled in the program than ^as to be 
expected from the number of applications and a significantly greater number of 
females were rejected than was expected. These findings reflect the intent \ 
of the program administrators to enroll a student bddy which was sexually bal- 
anced. ^ Since m^re females than males applied for the program, a greater 



Tiblc 41 
Chi Square 
Recruited Student Sanple Sex 



102 



df - I, a - .05» critical value • 3.841 
- expected frequency 
okaerved frequency 




\^5. critical valoc - 5.991 
k5.^^ critical value - 3.841 



to be rejected if the et^ollment was to be equal for 

rding the' race of applicants were complete only for the studetfcs 
the program. This information is presented in Table 43, Of 
th'ese 69 students, 78^per^t w,efe^Black and 22 percent wer\\7hite* Of the 
studeats who were accepted but declined. to participate in the program 55 per- 
cent were known to be Black and 32 percent were known to be White* 
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Table 43A 
Chi Square 
Recruited Student Sample Race 



Di3p08itlon 


^Black 


White 










of 


^ ^e 






to 






-Chi 


Students 


' chl-squafCv ^ 


chi square 






Sqaarc . 










Enrolled 


51.96 


5« 




15 








n - 69 . 


.0800 












.3242 

/ 


Drupped 


60.2^ 


51 


' 19.76 


29 






^^79 


n - 80 


1.^172 




4.3207 










Rejected 


\ 

88.85 


96 


29.15 


y 


/ 




\ 


n - 118 


.5753 




1. 7537. 

/ 








2.3290 


Total 
Ji " 367 


\ 

P -.753 \ 


p ^ .247 


8.39U 




- 2»a- .05» critical Vl'S 
d£ - 1 01- .05, critical val\ 

■ expected ^Vc<fuenc7 

■ observed frequency 
Chi Squire - / (f«-fo) 



Chi Square statistics were computed based^on the frequencies for which ^ 

^' ^ \ ^ ' ^ 

race was known. The Chi Square; results are presented in Tabl^ 43A. The over- 

***** ' - ^ . ' 

all Ch'r^Sqttare was Significant at the .05 level* The only-^individual 
comparison which proved to be statistically significant was the comparison of 
students who were accepted for the program but declined to enroll; the number 
of \^ites vjio were accepted but Reclined to enrpll was significantly greater ^ 
than wars expected. i 
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Attendance data for all applicants were compiled for* the previous 
year of school enrollment; information* regj^irding attendance is presented 
in Table 44. The Career Education Program* accepted. students whose absence 
from school the previous year ranged from no absence to 29 days of absence* • 
Much of the^attendance ^ata were unavailable; this was especially the case 
for students who were considered ineligible for the pro-gram; data were incom- 
plete for 91 percent of these students* The average absence rate was com- 
puted for 6ach of the groups based on available data. The average absence 
rate for enrolled students was 9^.41 days. The average absence rate for stu- 
den^(s wlko were accepted but declined to participate was 11.50 days. 



T«bl« 



^ R«cruiC«d Student Siffle: Attendance 



Studlnc 

Characteristic 


Accepted 


Enrolled 


Accepted/ 
Dropped 


V 

Rejected 


Total 


Days Absent 




X 




% 








X 




X 


24 - 29 dJtyt 


24 


15 


9 


13 


15 . 


16 ■ 


I 


.5 


25 






7 


18 - 23 days 


10 


06 

t 


S 


-7 


5 


5 


1 


'.5 


11 






3 


12 - 17 days 


10 


06 


- 5 


7 


? 5 


5 


0 


0 


10 






2 


6-11 days 


26 


16 


15 


22 


11 


12 




1 


28 






8 


0 - 5 days 


60. 


37 . 




46 ' 


28 


31 


' 13 


7 


73 






20 


Not >5iU^ble 


31 


20 » 


3 * 


5 


28 


31' 


183 


91 


214 






60 


Totnl 


161 


100 . . 


69 


^100 


02 


100 

_ 


200 


100 


3^1 


100 

o 


Mean , 


10.67 


9.41 . . 


^ 11.50 


5. 


68/ 


9.88 



r 



X05 

Chi Square analysis 6f this inforniatlon revealed no significant 
difference in the two groups for whom data were relatively complete; the prior 
school attendance of the students who were accepted and enrolled in the 
program was not significantly different from the prior school attendance of 
the ^tud^nts who were accepted for the program but ^clined to enroll. The 
Chi Square analysis for attendance is presented in Table 45, 



Table 45 
« Chi Square 

Recrjiced Student Sajaple Attendance 





Disposition of Studetits 


Days 0 


Earol led 

o 


Accepted/Dropped ' 


Chl 


Absent 


chi-«quare 


chl square 


Squa rc 


24 - 29 


12.19 9 


11.81 


15 




n • c24 


.8347 


.8616 




1.6963 


18 - 23 


5.08 5 


' 4.92, 


5 




n n 10 


.0012 


.0013 




;'0025 


12 - 17 


5^08 5 


4.92 


5 




n - 10 


.0012 


.0013 




.0025 


6*- 11 


^ 13.21 15 


12.79 


11 




n - 26 ' 


.2425 o 


,Z505 




.4930 


0 - 5 


30.48 32 


29.52 


28 




n • 60 


.0758 


.0)82 




.1540 


Total 
N • 130 


P ■ .508 / 


p • .492 

^ ' L 


2.4383 



df • 4, (A - .05, critical value « 13.668 
df ■ r» ^ - .05* critical value 3.841 
• expected frequency, 
■ observed frcquepcy 
Chi Square - y (^t'^^)^ 
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The Career Education Program accepted students whose previous grade ^ 
averages ranged from 60 to 100; these data are presented in Table 46,. Ba^a 
were incomplete for 40 percent of the studenjts rejected for the program, 8. 

percent of the students wHo were^ accepted Jjut declined to enroll, and 3' percent 

f 

of the students who enrolled. The previous ^rade average for the students 
who enrolled was 78.17; 48 percent of the students who enrolled had grades 
in the 70-79 range. The previous grade average for students who were 
accepted but declined to participate was 71.59; 53 percent of these, students 
had grade averages in the 60-'69 range. 



Table 

Recruited Stiident Saiaple Grades 



Student 
Char^icterlstlc 


* 1 

Accepted 


Enrolled 


Accepted/ 
Dropped ' 


' ' 1 ' 

lteject«d 


Totk^ 


Grades 


9 


X 




Z 


# 


\ X 






1 


X 


90 - MJO 


9 


' 06 


4 


6 


5 


5 


8 


^ 4 


17 




5 


80 - 8l 


36 


22 


21 


32 


14 






4 


44 






70-79 


51 


32 


34 


49 


17 




-36 


' 18 


87 




24 


^ eo - 69 


55 


36 


6 


10 


69 ^' 


y 


66 


' 33 


121 




33 


^ Not Available 


10 


06 


3 


3 


7 


y 8 


82 


40 


92 




25 


% Total 


•161 


100 


69 


ICO 


.92 


100 ^ 


^200 , 


lOO 


361 


100 




74.61 


78.17 


71.59 , ' 


70.97 • 


72.94 



Chi Square statistics wete computed for comparisons between the enrolled 

. ' ' / . " c • 

group and the group which was accepte^d but declined to enroll. The rejected 
studfents were -bmitted'^^om these ;comparisons because of the quantity of 
unavailable data; the^ data, are presented in Table 47»'' < 



♦ Tablf 47 

Chi Square 
Rccrultofi Sltudont S.icrlr i^r^.tp* 
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DisposUloa of Student* 




^- 


Crides 


'Enrolled 
chl-squire 


Accepted/Dropped 
chi sqoare 


" Chi 
Square 


90 — lOO 


3.93 


4 


5.07 


5 




n - 9 


,0012 




.0010 




.0022 


fin - »o 




22 




14 




0 • 36 


2.4992 




1.9395 




*4.4387 


70 - 79 


22,29 


34 


28.71 


17 




- 51 


6.1518 


♦ 


4.7762 




*10.9280 


60-69 


24.03 


6 


♦ 

30.97" 


49 




a - 55' 


13,5281 




10.4966 




x*24.4966 


Total 
N • 151 


P « .437 


p - v563 


39.3936 



df - 3, 0^ * .05, critical value - 7.815 

df » 1, • .05» critical value - 3.fl4l 

" expected frequency 

f " observed frequency 



Chi Square 



(f.-fo)' 



f 



^The number of students in the 70-79 and 80-89 ranges who enrolled was 



significantly greater than was expected. The number of students in the 60-69 
grade range who declined to enroll wSfe also significantly greater than was 
e^tpected. There was no difference between the two groups in the 90-99 grade 
range. It would appear, on the basis of th^ Chi Square analysis, that 
students were self-iponitpriixg decisions to enroll based on their grade averag 
this information Indicates ao^^ea .for further investigation. 



- » 
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Instructional Services \ * ' 

This part of the "Formative Evaluation Results" will describe the Instruc 
tional Services that have been provided so far this year in the Research for 
Better Schools Career Education Program. This section will-present informatio 
^egarding^Jiours-of instruction scheduled and attended, rates of attendance, 
grades earned in instructional activities, credits earned, and costs. This 
section will also present a description of the "typical" week for both lOtb- 
11th grade and 12th grade student^sr ^ 

A total of. 20,080 hours of instruction was scheduled-=^04i-JiLfi„5[j/th 
quarter.* Of the total hours scheduled, 7032 or 35 percen't^j^^re'^sched for . 
the Career Development Unit, 2314 hours or 11.5 percent were schedjii^ for the 
Career Guidance Ifeit, 6164 hours or 30.7 percent were schedU^e^^ the 
Basic Skills Unit, and 4570.hours or 22.8 percent wer^i^chedaL^d for the 



Supplementary Activities. For the lOth-lith gr^iie<giiiident:s , 6787 hours or 
33.8 percent of the totcil hours were sch€^aIedacro6s all activities; 13,293 
hour or 66,2 percent were scheduled for the 12th grade students. These data 
are presented in Table 48. For the. Career Development IJnit, 3226 hours or 16 
percent of the total hoiirs scheduled for all activities were^ scheduled for 
lOth-llth grade students; 3806 hours or 19 percent of the total were scheduled 
for lith grade students. For the Career Guidance Unit, 1254 hours or 6,3 
percent of^ the total hours scheduled for all activities were scheduled for 
lOth-llth grade students; 1060 hours or 5,2 percent were scheduled for 12th 
grade students, ^ For the Basic Skills Unit, 2307 hours or 11.5 percent of the 
total hours scheduled 'for all activities were scheduled for lOth-llth grade 
students; 3857 hours or 19.2 percent were scheduled for 12th grade students,. 
For the Supplementary Activities, no hours were scheduled for the lOth-llth 
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Table lii 

Hours o| Inatruccio Scheduled 
* rifth^mrcer 



Lcvrl 


Career ^ 

DcvelopcenC 


' Career 
Guidance 

Hrs. Z 


Skills 
Hr»r Z 


Supple- 
ocntsry 

Urs. Z 


TOTAL ' 
Rrs^ Z 


lOth-llCh 


3226 (16.0) 


1254 (6.3) 


2307 .ai.5) 
• 




6787 (33.8) 


12th 


38C6 (19.0) 


1C60 (5.2) 


3857 C19.2) 


4570 (22.8) 


13293 (66.'2) 


AU. 


7032 (35.0) 


23X4 (11.5) 


6lf6 (30.7) 


*570 (22.8) 


20080(100.0) 



* All percentage* boj»ed on hour«/tot*l hoxr» for all activities^ 

/ 



grade students since Olney Hi^h School provides that portion of their educational 
program; 4570 hours or 22.8 percent of the total hours scheduled for all 
activities were scheduled for 12th grade students. 

A total o^f 16,688 hours of instruction were ajt tended by the students in 

, ' ^ c 

the Career Educatign Program. For the Career Development Unit, 6204 hours of 
instruction were attended; this is 37.2 percent of the total hours attended for 
all activities. For the Career Gui'dance Unit, 1946 hours of instruction were 
attended; this , constitutes 11.7 percent of the total hours of instruction 
attended. For the Basic Skills Unit, 4395 hours of instruction were attended; 
this is. 26.3 percent of the total hours of instruction attended. For the . 
Supplementary Activities, 4143 hours of instruction were attended; this is 
24.8 percent o? the total hours of instruction attended. Tenth-llth grade 
students attended 5620 hours of 33.7 percent of the hours attended and 12th 

grade students attended 11,068 hours or 66.3 percent of the total attended. 

/ 

Information regarding hours of instruction attended is presented in Table 49. 



Hours of iDitrtctton Aciended 
Fifth Quarter 



< 


Lev«l ^S. 


Career 
Developoent 

Hr«. Z 


C«re^r 
Guidance 

Htm. Z 


Ski lift 
Hr«. X 


Supple- 
^ Rcntary 

Hr«. X 


TOTAL 
Hr«. Z 


- A 


lOth-llth 


2942 (17 »6) 


999 ( 6»0) 


167? Cioa) 




5620 (33.7) 


» 


12eh 


3262 (19.5) 


947 ( 5.7) 


27^6 (16.3) 


4143 (24*8) 


11066(66.3) 




ALL 


6204 (37.2) 


1946 (11.7) 


4395 (26.3) 


4143 (24.8) 


16668(100.0) 



* All percentasca bated on hourj«/cot«l hours for «11 activities 



For the Career Development Unit, lOth-llth grade students attended 2942 hours 
or 17-6 percent of the total hours attended by all students in all instructional 
activities; 12th grade st;udents. attended 3262 hours or 19.5 percent of the 
total- For the Career Guidance Unit, lOth-llth grade students attended 999 
huurs or 6.0 percent ^^f the total instructional hours attended; 12th graie 
students attended 947 hours or 5.7 percent of the total instructional hours 
attended. For the Basic Skills Unit, lOth-llth grade students attended 1679 
hours or 10.1 p^cent of the total instructional hours attended in all - 
activities;- 12th grade students attended 2716 hours or 16.3 percent. For the 
Supplementary Activities, no lOth-llth grade students attended since none were 
scheduled; 12th grade students attended 4143 hours or 24.8 percent of the 
total instructional hours attended. 

A 'total of 3392 hours or 16.9 percent of the total hours of instruction 
which were schedule^for the fifth quarter were not attended. Of the scheduled 
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instructional hours not attended, 828 or 24^4 percent occurred in the Career 
Development Unit, 368 or 10.8 percent occurred in the Career Guidance Unit, 
1769 or 52.2 percent occurred in the Basic Skills Unit, and 427 or 12.6 percent 
occurred in the Supplementary Activities • Tenth and 11th grade students did 
not attend 1167 scheduled hours of instruction or 34,4 percent of the .total 
riot attended; 12th grade students did, not attend 2225 scheduled hours or 65.6 
percent of the total scheduled hours not attended. Table 50 presents informa- 
tion regarding scheduled hours of instruction not attended. In the Career 

I 

0 

Development Unit, 10th and 11th grade students accounted for 284 hours of 
scheduled insrtruction not attended or 8.4 percent of the total and 12th 
'grade students accounted for 544 unattended scheduled hours or 16.0 percent 
of the total. In the Career Gu*idance Unit, 10th and 11th grade students accounted 
for 255 unattended hours or 7.5 percent of the total and 12th grade students ac- 
counted for 113 unattended hours or 3.3 percent. In the Basic Skills Unit, 10th and 
11th grade students accounted for 628 unattended scheduled hours of instruction 
of 18.5 percent and 12th grade students account;ed for 1141 unattended hours 
or 33.x percent of the total scheduled instructional hours not attended. In the 
Supplementary activities, 427 scheduled instructional hours were not attended 
hy 12th grade students; this constitutes 12.6 percent of the total scheduled 
instructional hours not attended;* 

Table 50 

Scheduled Hour* of Instruction Not Attended ^ 
Fifth Quarter f 



Vnlt 

Grade 
Level ^Nv^^^ 


Career 
Developoent 
Hrs. Z 


Career 
Guidance 
Kra. "Z 


Baalc 
SVlll* 
Hrt. X 


Supple- 
mentary 
Hr*. X 


* TOTAL 
Hra. X 


lOth-llth 
12th 


284 ( 8.4) 
544 (16.0) 


255 ( 7.5) 
113 ( 3.3) 


(i28 (18.5) 
1341 (33.7) 


427 (12.6) 


1167 (34.4) 
2225 (65.6) 


ALI. 


■828 't24.4)' 


368 (10.8) 


1767^(3772)"' 


427 (12.6) 


3392 (100.0) 



* .All perctntate* baaed on hours/total houra fur all i^ctlvitlea, ^ 
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The. overall rate of attendance for all students in all activities was 
83.1 percent. For the Career Development Unit, the overall rate of attendance 
was 88.2 percent. For the Career Guidance Unit, the rate was 84*1 percent. 
For the Basic Skills Unit, the rate was 71.3 percent and for the SupplementaiTr 
Activities, the rate of attendance was ^0.7 percent. The overall rate of 
attendance for lOth-llth grade students was 82.8 percent and the overall 
rate of attendance for 12th grade students was 83.3 percent. Information 
regarding rates of attendance is presented in Table 51. For the Career 
Development Unit, the rate of attendance was 91.2 percent for lOth-llth grade 
students and 85.7 percent for 12th grade students. For the Career Guidance 
Unit, the rate of attendance was 79.7 percent for lOth-llth grade students and 
89.3 percent for 12th grade students. For the Basic Skills Unit, the rate of 
attendance was 72.8 percent for 10th--llth grade students and 70.4 percent for 
12th grade students. For the Supplementary Activities, the rate of attendance 
for 12th grade students was 90.7 percent. Attendance for both 10th and 11th 
grade and 12th grade students appears to be reasonable for Career Development, 
Career Guidance, and Supplementary Activities . The attendance rate for all 
students in the Basic Skills Unit is a matter of some concern and should be 
investigated. 



Tiiblt SI 

lUtet of Attendnncr ^ 
Fifth <Hi*rter 











* «» 




Unit 


Career 


Career 


B«tlc 


Supple- 






Developaent 


Guidance 


Sklllrt 


iicnt«ry 


TOTAL 


Level 












Hrs. X 


Hrs. X 


Hr«» I 


Hr«» X 




lOth-llth 


91»2 


79.7 


72»8 




82.6 


12th ' 


d5.7 


89.3 


70.4 , 


90.7 


83.3 


ALL 


dd.2 


0 




90.7 


83.1 . 
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A typical weefe has been recoils tructured fdx a 12th grade student and a' 
lOth-llth grade student. 

A 12th grade student spends sik hours a week in the 'Basic Skills Unit. 
One hour a week is spent in the Career Guidance Unit. At least one day per 
week is spent in the Career Development Unit. Two hours a week are spent in 
one elective Supplementary Activity and another two to three hours are 
spent on a second or third elective Supplementary Activity.^ Three hours a'^ 



\ week ^re spent on a required Supplementary Activity. A "typical" roster for 



y:he week is presented below ♦ 





V 

h 


9:00 






Fencing - 


10:00 








11:00 


ILC 




Kath 


12:00 






Cuid;\nce 


1:00 


Reading and 


2:00 


Coaposit'ion 




3:00 


Perc&sclon 


A;00 




5:00 ' 





Flture 4 

"Typical" Roster for a 12tlv Grade Student 



ILC 
Math 



Typlnr 



Marketing 
Cluster 



Career 

Exp]orati6n 



Ksrdte 



ILC 
Math 



Typing 



.jKcading andi 
|C_opj>ot 1 ^}9^ 



Nevgpaper 



Since the lOth-llth grade student participates in courses and xextra- 
curricular activities at Olney High School, his schedule for the Car,eer Educa- 
tion program is different from that of a 12th grade student. The lOth-llth 
grade student typically spends five to six hours each week in the Basic Skills 
Unit, one day a week In the Career Development Unit, and two hours a week In 
the Career Guidance Unit^ With th^^e exception of the one day he spends in the 
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Career Development Unit, these activities are concentrated in the afternoon. 
A '"typical" roster for a lOth-llth ^rade student is presented below. 



9 

9:00 
10:00 
11:00 
12:00 
1:00 
2:00 
3:00 
4:00 
5tw 



Figure 5 

•Typical*' Rotter for a lOth-llth Grade Student * 
« T W ^ Th 




Guidance > 



English 



Transportation 
' Cluster 



Career Explora- 
tion 




^ A tot^Cl of 221.57 credits were attempted in th^ fifth quarter by all 
students enrolled in the Career Education Program.' Of these, 79.20 credits 
or 35.8 percent of the total attempted were in the Careet Development Unit; 
22.08 credits of 9.9 percent of the total attempted were in the Career 
Quidance Unit. Seventy (70.00) credits or 31.6 percent were attempted in, the 
Basic Skills Unit, and 50.29 credits or 22.7 percent of the total were attempte 
to earn 79.40 credits or 35.8 percent of the total and 12th grade, students 
attempted to earn 142.17 credits *or 64.2 percent of the t9tal credits. This 

52. In the Career^evelopnq^]^^ 10th and 
11th grade students Vt tempted to earn 38.50 credits a^.rr^A^perdent of the\re( 
attempted in all activf^s, while 12th grade, studertts attempted 40^7fi^redit\\ 
18.4 percent of the total. In the Career Guidance Uftit, 10th and llth graded 
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Credits Attevp'-ed 
Fifth garter 





^\Unlt 
Grade 
Level 


Career 
nevelopntent 
Kra. Z 


Career 
Guidance 
Hr». : 


Basic 

Skills 
Hra. X 




Supple- 
mentary 
Hra. Z 




TOTAL 
Hta. X 


















lOth-llth 


38.5 (17,4) 


13.0 ( 5,8) 


27.9 (i2.6) 




79.4 (35.8) 




12ch 

« 


40.7 (1».4) 


9.1 ( 4.1) 


42.1 (19.0) 


50.3 (22.7) 


142.2(64.2) 


c 


ALL 


79.2 (35.8) 


22.1 ( 9.9) 


70.0 (31.6 


50.3 (22.7) 


221.6(100.0) 



* Percentagea ^computed on credJta/total creditit for all actlvitlea. 



attempted 13.00 credits or 5,8 percent of the total and 12th grade students 
attempted t-.o earn 9.Q8 credits or 4.1 percent of the total. In the Basic 
Skills Unit, 10th and llth grade students attempted to earn 27.90 credits 
or 12.6 percent of the total and I2th grade students attempted to earn 42.10 
credits or 19.0 percent of the total. the Supplementary Activities, i2th 
grade students attempted. to earn 50.29 cr-edits or 22.7 percent of the total 
'^reattSL ^attempted in all activities. ^ , 

^ A total of 199.47 credits wei;e eatrned in tt(e fifth quarter in all 
activities of the Career Educatlttsi Program. Of t%se, 78.10 dr 39.1 percent 
of the total credits earned were in^e^Career Deveib^ent Unit; 18.08 credits 
oir 9.1 percent of the total were earned In the Career Guidance Unit. Ih the 
^asic Skills Unit, 55.90 credits or 28.0 percent of the totalw^re^arned; 
47.39 credits or 23.8 percent of the total were earned In Supplementary 




d llth grade students earned 74^^^0^credits ol;_37. 4 percent 
gracie studei^ts earned 124777 icredit&^^^r^^^^^ of 
:|on ^egarding cre^^t:€j|^| ^ in the fifth quarter is. ^ 





presented iV Table 53. In the Career Develppment Unit, 10th and^llth grade 
students eari^ed 38.50 credits or 19.3 percent of the credits earned in all 
activities and 12tK grade students earned 39.60 credit? or 19.fepercent of 
the total.' In. the Career Guidance Unit, 10th and 11th grW'^udents 
earned 10.40 credits or 5.2 percent of the total earned in all activities and 
12th'= grade students earned 7.68 credits or 3.9 percent of the total. , In. the 
Basic Skills Unit, 10th and 11th grade students earned 25.80 credits or 12.9 
percent of the total and 12th grade students earned 30.10 credits or 15.1 
percent of the total credits earned in alh activities. In the Supplementary 
Activities,, 10th arid 11th grade students earned no credit; IZtli, grade students 
earned 47*39 credits or 23.8 percent of the total credits earned In all 
activities. 

. A total^of percent. pf dll credits attempted were actually earned. In 

.( /•' ; ^ ^ \ ^ ^ . 

the Career IDevel^bpment Unit, 98.6 percent of ci^edlts attempted were earned, 4 
while in the Career Guidance Unit, 81.9, percent of credits attemptM^were 



earned. * In th^ Basic Skills Unit 



.9 percent of credits attempted were 



earned, and in the Supplemeiitars^ctivitifes, 94.2 percent of the credits 
attempted were, darned. Tenth\nd llth^^ra^e stHdents earned 94.1 perceyit of 



atl-credlts attempted and 12th grade students ear^^<Q7.8 percent of all 



* 



Table S3 

Credlti' E/irned 
^Flfth QuftTt^r 



^ y 



Unit 

Grade 
Level ' ^v;^ 


Career • 

DevelopoenC 

~ Hr». X ' 


^%^Coreer 


Boaicf^ 
Sklha . 

Ur*. X 


Supple- 
nentary 

Hra. X ^ 


TOTAL 
Hra. X 


lOth-lIth > 

\* ^ - 


^ A. <■ 
39.6 (19.8) 


7.7 ( 3.9) 


\ 

30.1 (IS.l) 


A7Uj23.a) 


74.7 (37 i4) 

< 

124.8 (IS2.6) 


\ 

MX 


\^8aH39.1). 


18.1 ( 9.1) 


55.9 (28.0) 


A7.; (23.8). 


— , 

199*5 (100.0) 




*J?<jrccntotoa co»pu><d on credits/total cedlta ftor all actlvltlea 
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redlts attempted. Table 54 presents information regarding the percent of 
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^redlt 



credits attempted that were earned. 



Table 56 

Percent Credit* Attefcpted Earned 
Fifth Quarter 



Cradc 
Level 


Cftreer 
Developnenc 
Hr«. Z 


Cnrecr 
Guidance 
Hrs . Z 


losic 
Skills 
Hra. Z ' 


Supple- 
nentary 

Hr». Z 


TOTAL 
Hra. Z 


lOth-llth ' 


100.0 


-\ 
80.0 


92.5 


IT- 


94.1 


\l2th 

\ 

\ 


97.3 




71.5 


94;2 


87.8 




98.6 


81.9 


79.9 


96.2 


90.0 



\ 



In the Career Development Unit, lOt^ and ll^h grade students earned 100 per- 
cent of the credits attempted and 12th grade students earned 97 . 3*p6rce^nt of 
the credits attempted. In the Career Guidance ^Unit , 10th and 11th grade 
students earned 80.0 percent bf the credits attempted and 12 tH grade students 
earned 84.6 percent of»the credits attempted. In the Basic. Skilly Unit, 10th ^ 
and 11th grade students earned 92.5 percent of the credits attempted and 12th 
grade students earned 71.5 percent of the credits attempted. \:n the S^pplemen- 
tary Activities, 12th grade students earned 94.2 percent ofW%he credits 
attempted. Percent .of credit^ attempted earned is reasonably high for 10th 
and 11th grade students in the Career Development Unit and the^B^sic Skills 
Unit; the percent of credits earned ife low for lOth and 11th grade stude^s in 



the Career Guidance Unit. Percent of cred'l^s^earned is reasonably high for 
l?th grade students in the Career Development Unit and tiie Supplemen^ta^. 
Activities; it is low in the Career Guidance Ui^it and the Basic ^Skills Unit-: 
The poor rate of achievement in the Career Guidance. Unit -may be due to the 
low credit value assigned to these activities (. 2 crecjit per cjua^er for lOth 
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and 11th grade students ^nd .1 credit per quarter for 12th ^^^e students). 
There is no apparent explanation for the. low r^^i^er^f (^^^i^evement by 12th 




grvade students in the Basic ^MlisJJnrt,other than the low attendance rat^. 

The ovejiall average grade for all^actiAtlties in the fifth qi 
2.44 on a scale from 0 to A/^^ftV A = 4, B J, C = 2, D = 1, 1^ 0. In the 
Caree^Development Unit, the av^r€ge^^a4i^ was 2.56 or C+/B-. I For the Basic 



_Jk^^ Unit, the average grad^was 2.28 or slightly above C. 



the 



Supplementary Activities the average grade wal a 2.90 or about a B. There is 
no average grade fpr the Career Guidance Unit since the gra^ding system used in 
^hose activities was Pass/Fail. The a.v^age .grade f qj. lOf h and 11th grade 
students in the fifth quarter^ wa^ 2.39 or a C:K the average grade for 12th grad 




studenjt-s was^.46*or a C+. Information^ifegaj'ding avetage grad^^& is presenjped 




Tab^^J55: In tKe Career Development Unit, the avera^^gra^ie^^for 10th 
grade students was 2.3i^>0T about a the av;^rage^<^e^or l^lf^grade 




* B«#<d^^jm^O - 4 scale : A»i. C-2, D-l, F-0 



5nal services^roVi'ded during, the fifth quarter 



,080Ji€rars of instruction were scheduled and 





)f attendance 




A review o 

a total 

that 16,688 J^ifi^s instruction were attended* Rates of ^ttenda, 
within JJi^ units of the cWeer Education Program: 
^exe: reasonable for the Career Development Unit, the Career Guidance Unit, 
and^^e Suppiemept^ry ^A^^^!9e^^^and well below expectation for the Basic 
Skills Unit/ Students enrolled in activities which reprQ3ent a total 
credits arid actually earned 199,47 of those credits. Rates*^ of eajiSng^redits 
were above-SD percent for all but 12th grade students in thjj^sic Skills Unit 
the rate of earning credits is reflected in the grade;;:^ragats of the students 
irythe various activities; grade averages \feve jk]^^--(^ level or abbVe for 10th 
^and 11th grade students for alj/activitiesf-^^elf th grade students has a 
or B- average in all acti^±i;A&i exc^t the Basic Skills^ 1^t inhere the^ 
had a C average 



Cost of Instructional Services > - ■ 

'This section of the "J^c^rmative Evaluation Results" will present .various 
cost analyses relating**to the RB§ Careex^Education Program, Comparisons will 
be, drawn regarding the relative costs withiti the Career Education Program and 
with |he Philadelphia School District," <:oinparisons with the S^:hool District 
will be limited to those for which figures are^^3«rilable . ' - * 

. Thd costs of .the Career Educatipiv--Ffogram"'x^ere computed in several ways: 
actual^ expenditure, ^ o §t I5er student, 4:;ost per hour of instruction scheduled, 
cost per^hour'^of^instructlpn attended, cost per credit attfmpt^dy ^artd'Wst''^^ 
'Cfedit earned/ The cost information is pres.eat^'tn Tablets 56 through 62* 
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The total expenditures for the Research for Better^ Schools Career Education 
Program for the fifth quarter were^ $91,148.00. This figure includes a $50,000 ' 




advance on a subcontract to the Greater Philadelphia Chamber of Connnerce. T 
obtain a more realistic figure for the purpose of computitig other in-formationj^ 
"it^was decided that the average quarterly expenditur^forthe f if ^ and si 
quarter should be used. The average expenditjuife p>«^ quarter was $80^120.50 
Ail. actual expenditures listed in th±s r^ojp€^re based an the average quarterly 
expenditure. The actual expenditure for 'the Career Development Unit was 
$37,867.00 or 47.3 percent of the total expenditure. The actual expenditure far 
the .Caxe§^si^,^^^^^^^^it was $18,638.50 or 23.3 percent of the total. The 
^ actual 'expenditure for the Basic Skills Unit was , $12, 086. 50 or 15.1 percent of 
^t^ie--W0j7''The actual expenditure for the Supplementary Activitid's^was 
$11,5^8.50 or 14. 3 percent of the totajl.^ The actual expenditure for 10th and^ ' 
ILCh gra3^- students was $31,955*62 or 39 .*9 percent of the total. The actual^^;^ 
^xpeniii-ture for 12th t^ade students was $48,164.88 or $ 0.1^& ercent of the total 
^Table 56 presents ^formation regarding the actual expenditures in the Career 
Education Program. In the Career Development Unit, the* actual expenditure was 
^$07,418.82 or 21.7 percent of the total for 10th oxv^^lVCc^ grade students and - 
$20,448^8-'i5f""2?r6 per\:6Vit of the total for 12th grade students. In the ^ 
Care^r^idance Uait, the actual expenditure was $10,064. 79^r 12.6 percent of 
i the total fpr 10th and 11th grade students and $8j^5^*71 or 10.?^percent of 
the, tofal for 12th grade students. In thg,^-B^ic Skills Unit, the actual 
expenditure w^s $4,472.01 or 5.6 percent of the total for 10th and 11th grade 
students and $7,,614.,49 or 9.5 percent of the totail for, 12th grade students. 
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supplementary Activities,, fhe actual" expenditure fox 12th grade students 
was $1J.;52S.50 or 14,3 percent of the totals ^ ' . t . 

Table 56 ■ 

j5o«c: Average Quarterly Expeaditurc Fifth and* Sixth Quarter -T'^ 




\v Unit 
Grade 

J45t£i j:^^ 


' Career 
Dev«lop»ent' 


Career 
Guidance 

SO.OO X 


^.islc 

;kiii» ' 

SO.OO TL^ , 


Suppletaentary 

-^.00 I 


soVoo r 


lOth-llth 

^^^^^ 


^^^"^^ ^ — 
^^^^^^^^^^^'^^^'^"'^^'^ 


10.064.79 12j6 

^^iSn ' tcr.7 


,.^*f5i72r5i ' 5.6 

7^,$J5>^^ >.5 


11,528.50 U.5 


31,955.62 39^.9 

1 

48,164.88 60.1 




^^^tToO^ 47.3 


18,638.50 23.3 


12,D86.50 15a 


11,528.50 U.3 


$80,12Q.50 100 



The aversige^osfe' per student for the 



^^iftli quarter was $602.07; thip cost 
is a weighted cost"'per-^rPdent~^ pA^r ticipatlon in supple- 

mentary activities. -The average cost-per student for the core program only ^ 
(Career Development, Career Guidance, and Basiq^ Sky Is) was $511.88 for the ^J..- 
f if th^ quarter. The cost per student in the Career Development Unit was $282 ; 59.- 
^he^^t per st^udent in the Career Guidance Unit was $139.09.^ The cose per 
student iiT^the Basic Skills Unit was $90.20. The cost per student 'in the 
Supplementary Activities was $177.36. The average cost per 10th ancj 11th grafie 
student was $463.13. The average cost per 12th grade student was $741. OQ 
Table 57 presents information regarding the costs per student. In the Career 
nevelopment Unit, the average cost per student was $252.45 for 10th and 11th grade 
students and $314.59 for 12th' grade students. In the Career Guidance- Unit , the 
average cost per student was $145.87 for 10th and 11th grade . students and 
$131.^0 for 12th grade students. In the Basic Skills Unit, the average cost 
per student was $64.81 for 10th and 11th grade students and $117.15 for 12th 
grade students. In the Supplementary Activities, the average 49st per 12th 



■S.40 



Table ^7 
Cost ?er Student Fifth Quarttr^ 
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iMlt 












Grade 


Career 

Developnent 


Career 
Guidance 


ftatic 

' SlclUa 


Suppleaentary 


TOTAL 


Uvel \ 


$0.00 


^$0.00 


SO.OO 




fift.nn 


lOch-llth 


252.45 ^ 


U5.87 


64.81 




A63.13 


12th 


3U. 59 


131.90 

t 


117.15 


177.36 


741.00 


. ALL ^ 


282.59 


139.09. 


90.20 ' 


- 177.36 


S602.O7* 



♦ CofC per atudent In core prograo (without Suppleaentary Actlvltlea) it $511.88. $602.07 U weighted coat 
per atudent to allow for 12th grade participation In Supplementary Actlvttlea. ^ 

grade Student .was $177.36. The cost per 12th grade student was higher than 
the cost .per 10th and 11th grade student for the Career Development Unit, the 
Basic Skills Unit, and the Supplementary Activities; the cost per 10th and" 11th 
grade student was higher than the cost per 12th.^ade student for the Career 
^GuiHance^'toitT The total average cost per 12th grade student was much higher^ 
than the total average cost per lOth and 11th grade student; most of this 
discrepancy is accounted for by the providing of Supplementary Activities for 
12th grade students only. When tb^cost for the Supplementary Activities is 
removed> the cost per 12di^^^^ is $100.51 greater than the cost per' * 

10th and 11th grade sttident. This^ differeni(5^ partially reflects the greater 
individualization needed for students participating in Career Development 
SpecTializations and Basic Skills Independent Studies. 



he-averaige cost per hour of instruction scheduled was $3.99. The average 
cost per hour of instruction scheduled was $5.38 for the Career Development 
Unit,. $8.05 for the Career Guidance Unit, $L.96 for the Basic Skills Unit, 
and $2.52 for the Supplementary Activi*tie^. The average cost per hour of 
instruction scheduled was $4.71 for 10th and 11th grade student and $3.62 for . 



12th grade students. Table 58 presents in format ion. regarding the costs per 
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Table 5rt 

Coit Per Hour of In»l.*ucclon Scheduled Fifth Quarter 



Unit 

Grades 

Uvel \^ 


Career . 
Developcsen^ 
SO. 00 \ 


Caree^ 
' Guidance 
SO. 00 


Basic 
Skill* , 
•So. 00 


Supplementary 
SO. 00 


1 

TOTAL 

so.bo 


lOth-Uth 


S.40 


8.03 


1.94 




4.71 


" I2th 


5.37 


8.08 


1.97 


2.52 


3.62 


ALL - 


5.3« 


8.05 - 


1.96 


2.52 


3.99 



: 4 



hour of iastruction scheduled* In the Career Development Unit, the cost per 

hour of instruction scheduled was $5,40 for 10th and 11th grade students and 

$5,37 for 12th grade students. In the Career Guidance Unit, the cost per hour 

of instruction scheduled was $8,03 for 10th and 11th grade students and $8,08' 

for 12th grade students, ^ In the Basic Skills Unit, the cost per hour of 

instruction scheduled was $1,94 for 10th and lltji grade students and $1,97 for 

12th grade students* In the Supprementary; Activities, the cost per hour of 

instruction scheduled was, $2* 52 for 12th grade students. The cost per hour of 

instruction scheduled was about equal for 10th and 11th and 12th grade students 

for;:a:ll activities for which both groups were scheduled, .The cost per hour 

o'f instruction scheduled was highest for the Career Guidance Unit, next highest 

for the S^upplemetitary Activities, and lowest for the Basic Skills Unit, The 

relative standing of these cdsts partially reflects the intensity of staff 

involvement necessary for the programs to be implemented. Already prepared 

materials are used in the Basic Skills Unit, The Supplementary Activities had 

relatively, high enrollment in the required activities. The Career Development 

• • • 

Unit consists of explorative activities by relatively small groups (8 to 10 
students at the largest). The Career Guidance Unit primarily provides , small 
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small group or individual^ counseling* The nature of the activities is readily 
reflected in the relative costs^ per hour of instruction scheduled* 

The average cost per hour of instruction attended was $4.80. The average 
cost per hour of instruction attended was $6*10 for the Career Development Unit 
,$9* 58 for the Career Guidance Unit, $2.75 for the Basic Skills Unit, and $2; 78 
for the Supplementary Activities. The average cost per hour of instruction 
attended was $5.69 for 10th and 11th grade students and $4.35 for 12th grade 
students. Tab-le 59 presents information regarding the costs per hour of 
instruction attended. In the Career Development Unit, the cost per hour of 
instruction attended was $5.92 for 10th and 11th grade students and $6.28 for 



Table 59 

Co»t fer Hour of In»t.ructlon Attended Fifth Quarter 



"N^^ - Unl t 
Grade 


Career 
Developtcnt 


Career 
Guidance 

SO. no 


Basic 
Skills 

SO. an 


Suppleioeatary 

^o.nn 


TOTAL 

<sn An 


J^Yp] — ^ 

lOth-Uth 
12th 


5.92 
6.28 


10.07 
9. OS 


2.66 
2.80 


, 2,78 


• 5.69 
A. 35 


ALL 


\ 6.10 


9.58 


2.75 * 


2.78 


^.80 



12th grade students. In the Career Guidance Unit, the cost per hour of instruc 
tion attended was $10.07 for ICth and 11th grade students -and $9.05 for 12th 
grade students. In the Basic Skills Unit, the cost per hour of instruction 
attended was $2.66 for 10th and llth grade students and $2.80 for 12th grade 
students. In the Supplementary Activities, the cost ^er hour of instruction 
attended was $2.78 f9r 12.th grade students. The highet costs per hour of 
instruction attended reflect the- rates of absence in each activity. 
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The* cost of absenteeism* to the Career Education Program was compjtitedV^y 
multiplying the number of hours of absence for each activity by the cost per 
hour of scheduled instruction. The total cost of absenteeism to the program 
was $11,957.70 or 14.9 percent of the total expenditure for the Career Education 
Program in the fifth quarter. The cost of absenteeism was $4,454.88 or 37.2 
percent of the total for the Career Development Unit, $2,960.69 or 24.8 percent 
of the total for the Career Guidance Unit, $3,466.09 or 29.0 percent £or the 
Basic Skills Unit and $1,076.04 or 9.0 percent for the Supplementary Activities. 
The cost of absenteeism by 10th and 11th grade students was $4,799.57 or 40.1 
percent of the total. The cost .of absenteeism by 12th grade students was 
$7,158.13 or 59.9 percent of the total. Table 60 presents information 
regarding the costs of absenteeism. In th? Career Development Unit^ the' cost 

Table 60 

Cost of Abficnceeiea 
, Fifth <^urcer 



\v Ul'.lt 

erode 

Level ^ss,^^ 


Career 
DcxAlopnent 
$0.00*^ z 


Career 
Guidance 
$0.00 X 


Biisii; 
Skills 
$0.00 X 


Supplcnentary 
SO.OO * X 


TOTAL 
$0.00 Z 


10th - 11th 
12th 


1,^33.60 12.8 
2,921.28^ 24. ^ 


2, 0^7.65 17.1 
913.04 18.8 


$ 

1.218. '.2 10.2 
2,247.7T^18:d 


1.076.04 9.0 


4,799.57 40.1 
7,158.13 y 59.9 




4,454. 8« 37.2 


2,960;69 24.8 


3,466.f9 29.0 


1.076.04 9.0 


11.957,70 lOQ.O 



* All percentages bssed on cost/total cost iox «11 sccivities. 

of absenteesism was $1,. 533. 00 for 10th and 11th grade students and $2,921.28 
for 12th grade students. In the Career Guidance Unit, the cost of absenteeism 
was $2,047.65 for 10th and 11th grade students and $913.04 for 12th grade students.. 
In the Basic Skills Unit, the cost of absenteeism was $1,218.32 for 10th and 
11th grade students and $2,247.77 for 12th grade students. In the Supplementary 



Cost of absenteeism is not a real. cost; rather it is an artificial statistic 
which, estimates the cost of .services not used. 
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Activities, the cost of absenteeism by 12th grade 'students was $1,076*04* 

The cost per credit attempted in all activities was $361. 60 • The cost 
per credit attempted was $478.12 for the Career Development Unit, $844.13 for 
the Career Guidance Unit, $172.66 for the Basic Skills Unit, and $229.24 for 
the Supplementary Activities. The cost per credit attempted was $402.46 for 
lOth and Xlth grade students and $338.78 for 12th grade students. Table 61^ 
presents information regarding costs per credit attempted. In. the Career 



Table 61 

Co»t Per CrcdlC Atteapted Fifth Quarter 



Unit 












Gr«do 


Caret r 
iJave lop tent 

$0.00 


Career 
Guidance 

$0.00 


Baalc 

Sklllf 

SO. 00 


Scpplcmtntary 

fio.no 


TOTAL 


lOth-llth 


452.44 


774.21 


160.29 




402.46 


12th 


S02.41 


944.24 


180.87 


229.24 


338.78 


ALL . 


^78. 12 


844.13 


172.66 


229.24 


361. $0 



Development Unit, the cost per credit att;empted was $452. 33 for 10th and 11th 
grade students and $502.41 for 12th grade students. Inthe Career Guidance 
Unit, the cost per credit attempted was $774. 21' for 10th and 11th grade -students 
and $944*24 for 12th grade students. In the Basic Skills Unit, the cost per 
credit attempted was $160.29 for 10th and 11th grade students and $180.87 for 
12th grade student^^. In the Supplementary Activities, the cost per credit 
attempted by 1.2th gr^de students was $229.24. 

The average cost^^per credit earned was $401.^67. The cost per credit earned 
was $484.85 for the Career Development Unit, $1,030.89 for the Career Guidance 
Unit, $216.60 for the Basic Skills Unit and $243.27 for the Supplementary 
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Activities. The c«st per credit ^§§^tt!^i^^^IX--aa^^ studeo^s ,was 

$427. 79; the c^st^T^^.^^^ by 12th grade*^irdents was $386.03* 

Table 62 presents infornfe^^ion regarding the cCrs<s per Credit earned. In the 



Table 62 

Coster Credit Earned Fifth Quarter 



\. Unit 
Grade 

Uvel \ 


Career 
Developaent 

SO. 00 


Career 

Guidance 

so.oo 


- \^ 
Baaic 

SkllU ^ 

•50.00 


Supplectentary 


TOTAL 

50. nn 


12th 




967.77 


173.33 




427.79 
3«6.03 


516.37 


1116. 37^\ 




243.27 


ALL 




1030.89 


2ie\60 


^243.27 ^ 


^''"■"^4^^?67^^* ^ 



Career Development Unit, the cost per credit eai^ned was $452.44 for\J.Oth and^~ 
11th grade students and $516,37 for 12th grade stadeh^tTr^^In the Career 
Guidance Unit, the cost per credit earned was $967.77 for 10th and llth grade 




students and $1,116,37 for 12th"grade Students. In the Basic Skills Unit, the . 
cost.per credit earned was $173.33 for 10th and 11th grade student^ and $252.97 
for 12th grade students. Ixx the Supplementary Activities, the cost per credit 
earned was $243.27 for 12th grade students. 

The only data which were available from the Philadelphia School District 
for purposes of .comparison were those relating to cost per student per year. 
l?he cost to the Philadelphia School District of providing instructional services 
to secondary students was $1,234.83. This figure was computed by subtracting 
administrative and debt service costs from the total operating budget and 
multiplying that figure by the percentage of instructional services directed 




-education* Comparab.le figures h? 
1 Qareet Edueatio^^^gram. Based on^an extension 
I the fiCfth and sixth quaH^s^_/the cost,^.^i-^^ix^v± 
\ s^ervices to a 10th. or 11th gra^student is $l,3B9r.52; the 



of instructional services is^^i^g^u^S^'^^f^^ or 11th grade student 

The cost of providing 9 months of instructional services to a IZth grade stud 
in the"lSreer Education Program is $2,?23.00i th^Q§t^of providing, tji^se 
services for 12 months is $2,964rO0., Since orrly 12th grade students receive 
their entire educational program frgm the Academy^ for Career Education, that 
figure should be compared to the Philadelrphia School District costs; for 
9 months of instructional services, the compari^n is $2,223^.00 for the Career 





- Education Program versus $1,234.83 for the Philadelphia^^^ool District. The 
cost of providing a Career Education Program^ is 1.8 times the Pttiladerphia 
School Dj^s-trict's costs of providing educational services to the same Ifev^l" 
students. i ^ . 

A goal of the Career Educatibn Program for this year was to reduce the . 
per pupil cost of instructional services 3C^pi^cent ta^reach a $3,309.36 cost 
per student for the year. CdSts incurred so far this^^yearXndicate that that 



goal will be met and surpassed. The cost of. providing instruct 
listed below: 




vices is 



lOth and 11th Grade Students 

Cost per student fotL 9 months . .-. $1,389.52 

Cos^T^er student for 12 months 1,852V52 



; erJc 





grade stu^nts^wiTr^^icipate in the Career Education Program durin^^;;t^ 
Summer P^t;he cos^^ j)rovi^l|ig instructional services to eacKlof^ those students 

In 




wilL be :the IzTnofT 



In Dath ca^es, the figures are 



well below the anticipated co^t^ner student of $3,309.36 




Summary* ^ 

This section of ^he^eport has described the Career Education Prqgcaa and 
presented formative evaluation results regarding the recruitment^atii^-s^ect^on 
of employers and site analysis, the support systems of the /rogram, the use of . 
advisors,' the recruitment and selection of students, the instructional services 
provided to- students, and the cost of providing those instructional services. 

The Research for Better Schools Career Educatior 
elements: the Career Development Unit, the Career Guid£ 
Unit, and Supplementary Activities. The first three ai 
'in the program; the last is limited to only 12th grade 

School provides those aspects of the program to 10th and, 11th grade stu^fe^its. 

Thesje^activities are provided^-b^j^ ' full-time pa:qf essional staff. The &^aff 
fall into /our c^atego^ies: Program Adm£Mstrat^n^(l77^5valaiai^^ Design 'and 

Development (13), and Operations (12). Staff were ^viewed as being capable of 




\ 



I 14R- 
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^ prog^» The only staff areas in need of 



delivering all components of the 
^-^^£"^ther development were the Career Development and Career Guidance Units,. 
Where'-l^iterrelationships among the Chamber of Commerce Staff, the team leaders 
of the Unib^^nd the Project Direct'or' of the. Chamber Staf^ were perceived as 
needing clariri^ati^tv^nd definition- Training was minimal due to the late 
date ol^Ttneii^atio^f the^on^ract vith the National Institute o-f Education* . 

Facilities used i^i the Career Educal>iQxi^rogram embrace four major sites: 
The Academy for Career Education, the RBS offices, the Greater Philadelphia 
Chamb^ o^ Commerce, and employer sites • The facilities are adequate for the 
•needs ot "tt«*^urrent enrollment. 

TTie""Tne^uitilt8^t and selection of employers is based on a clustering of 
"-"^mpl nynrrNj*4:;i^ areas of related activities. Last year there were 12 clusters; 



^s have^^een added to the ^i^rpgram, representing a 33 percent 

/ 



incrfe^ in thiss,type of activijy. A total of 84 employers has been recruited 

and selected f orl5flrt«it>ation in the Career Education Program in the last two 

years. Of J^ese, ,53\ere recruited last year and ^1 were recruited this year. 

Of the, 53>artieS^ting in FY 1973, 14 did not participate in FY 1974. Reasons - 

for non-par ticipati^ include: the activities were not adequate, funds or staff ' 

^ \ ■ • • • . ] 

were unavailable, studem;^ ^ere perceived as insufficiently motivated, and the 

^ employers^vere in thV.prdce^of internal reorganization. An- additional' 9 

employers have withdrawn from thfe program in FY 1974. Reasons for these with- 




clu dej the activities provl^^ ed were not adequate, funds' ^^r staff , were 
not available, and involvement l>y^'the organization in a major building campaign* 
^oT^tFr^^-TO-ei^^ who have p^rtici^ted in the FY 1974 Career Education Program, 
61 or 87 percent remaina^rbi^t^ly involveik^ and interested in the program. 

Policy making for tn§ Career MttG^ibn Program is vested in the Board of 



Directors o^^^die-^AtfaJen^ for Career Education. The composition of this board 
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includes representatives from industry, labor, education, and th*e community. 
Most advisory groups have been incorporated into the decision -making process by 
granting represetitatj on on the Board of Directors of the Academy, In addition 
to traditional advisory groups already having membership on the Board, the o^Jer 

V 

of Board membership has been extended to parent and student groups. Other * 
advisory groups include NIE, jthe other Experience- Based Career Education Programs, 
and the Board of Directors of Research for Better Schools, 

Student recruitment and selection in FY ^^74 consisted 'of the selection 
and enrollment of 10th and 11th grade students wJi^s^ould participate, in the core 
aspects of the Career Education Program (Career Development, Career Guidance, and 
Basic Skills) and receive the supplementary aspects of their program from their 
sending school. A total of 69 students Were accepted and enrolled in the 
Academy ^or Career Education Ptogram. An additional 92 students were accepted 
but declined to participate in the program, A total of 200 students were 
considered ineligible for tjie program. The greater portion of students accepted 
^werd 11th grade students ^ The rate Df acceptance was about equal for each sex 
although more males than females enrolled. Few Whites applied for the program, 
arid as a result 80 percent of the enrollees were Black. If ^ this racial tren^ 
cba^nue^^the Career Education Program will fiave served metnbers of only one 
population subgroup • Analysis of data on previous grade averages indicates 
thai! students with 70-89 averages who are accepted for the program decided to 
enroll, while students who wete accepted with 60-69 averages declined .to, enroll. 
Two factors of th^ student recruitment process which warrant iEiurther investigation 

the apparent self-monitoring of decisions to enroll ^ased 




A review of the instructional services provided d^aring the fifth quarter 



shoyjs th-at a total of '20,080 hours of instruction were scheduled and that 



16,688 bouFs of |nstructioiv^ere attejuded. Rates of attendance varied sharply 
iTlthevunits of tjae^^^reer Ed^ucation Program. Rates of attendance .were 



reasonable for the Cateer^Developinent/^mit , the .Career Guidance Unit, and the 

€11 below expectation for the Basic Skills Unit; 
ies which represent a total of 22.157 credits a^kl^' 
redits- Rates of earning credits were above; 
tudents in the Basic Skills Unit; there the 




71.5^ercent. The rate of earning credi^s^s reflected in the^ grade 



of students in theJi^iSus activities* Grade averages for 10th and 
students were at or ^b^ove.a C+ level for all activities. Twelfth grade students 
had a B~ or B- average in ail activities except the Basic SkiXls Unit j^Jpare they 



80 percent 
.rate was 

averages 
11th grade 



had a^C average, ^reas of particular concerp are the rate of earfiing of credi^ 
'by 12th grade studa^^^itf^he Basic Skills Unit and the rate^of attendance by 
all students in the Basic Skills Unit. 

The quarterly-cost of providing instructional servicerg^was $602.07 per 
student; this is a weighted cost which accounts f-or 12tj>Xgrade p^rxicipat^-qn in 
the* Supplementary Activities. T^he quarterly cost per student^^^^pSf the core 
program only was $511.88. The cost per credit earned was $401.76*, Total oper^- 
tional expenditures in the Career Education Program in the fifth quarter were 
$91,148; this figure includes an advance of $50,000 on a subtontract for instruc-^ 
tional services. The average quarterly expenditure for the fifth and ^ilcth 
quarters was $80,120»5Q. Projected costs for providing 12 months of instructional 
services to lOthand 11th grade students are $1,852.52 per student. Prgj^cted 
costs for providing ^ months of instructional services to l^h grade students 
-graduating in June 1974 are $2,223 per student. Both^tliese figures are below 



the anticipated yearly cost per student of $3*, 309/36 



133 * 



X. DISCUSS-toiT AND. RECOMMENDATIONS 



This last section of tt ^Itrferim Evaluation Report ^ Intended to dis- 

cuss^lr^^m^^ issues and phenomena noteijLn_4Uia-^ 

above. "In some cases it is attempted to interrelate information from several 

sections. Topics wete selected for treatinent in this section based ^on^their 

perceived import for the future of the program, the clarity^itlTwhich: they 

* ' - , . . ' " ^ 1/ ^ 

pharacterize "the present" program, or the themes which they suggest for further 

* * - ' ^, " 

research and dejvelopment' in Career Education. 'This section is'TTess formal t 




its predecessors; reflection .and speculation are admissible as -interpretive 
techniques^ At the conclusion of each topic-"of discussion a reconp^datioh 
derived from that discus^^n is presented. * 
Interim Eva34i4t;ion Rejx^ts 

Thi^::: :interim Evaluation Report required^ a p^oxima^^ j ^o^m^^rmoftthg 
ol^tS^t to complete. It is essentially affinal reper£;-with 
of final results. The imp or tan <fe^ of \f^n% t^-s-'^e^y^^^:m^^n ^ to the 




llest extent of the-'^alvwhich be. 



ii understood- and 

^predated within the context, of JiXE ^^mOiilg corlsiderations. However, it. ^ 
^lould also' be understood that Ehe^^mi^>^^ period is-tfe^st ^emanding^m^ 

in the conduct of * programepjjili^^i^ competition for resources 

^^^^^^ ^ 

bej:w^n providing inptit^ for next' yea^^ funding and. success illy completing* 
this year*s>\^uation effort, tlie interim report should be focused as much ^ 
as^^ppssible on those issued deemed by ,NIE to be most critical in considering 
future fundin^^ This would result in a 'report which could distuss key issues 
more fully arjd ^eliminate program facets which may be interesting Ifut imight *^ 



arjd^ 
4^ 



not warrant tR)p 3fesourc6s necessary for explication ,ajt^ inid-ye^rf^^li/ interim 
report could possibly be more useful and manageable. *' Individual program 



\0 
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evaluators could make the suggested decisions to select content, but it 
— 4«)uld be more appropriate for NIE to do so in its guidelines. This year's . 
guid^lijies were well constructed and all indlusive; it is suggested that perhaps 
thejr were toa inclusive for the task at han^. ^ 

Recoimnend^tion 1, Prior to funding next year, NIE should determine the 
content and extensiveness of major evaluation reports which <&re to be used for 
^ management ^nd planning purposes. This would better enable the J^orporation 

of NIE's irjjti^sts into evaluation planning, and would r^stift in more focused 



and us^le evaluatj 



>rLy bucti planning would /have been difficult in the 




^^^t, but, as jthe-'pfograms have achieyed mote resolution, such action has 



become more' feasible. 



Re^knm< 



/ 



end a ti on 2> After/the i^a^iure and exter 



t of repor*tipg 1iave been 





determined, a realistic level of r^iiurcea should be allocated in the budget 
fp'^ the effort requi^red:" 
y Student. . Populat^ions • , 

It is apparent that^the total of students availlable for analysis(261) 
is sufficient for extensive research and evaluation ooncerning student effects. 
Hovfever, the c6mp<^si^ion of the groups comprising this total present^ some 
limitations.' 'The ACE Sroup has been drawn from the ciVy-wide secondary school 
populati^, while all other groups have been drawn ^om\a single school, Olney 
High. School. Also, the ACE Group received a comprehensive educationlfl program' 
'from the Acadeijiy while the other experimental group, ACE-Dlney , received a core 
career edjLitfation program from the Academfy and other currictilar and extra-curric- 



:^r-o££ering5^Jro^^ Either one of these\dif f erences alone' 

jwould make possible an Additional controlled comparison,^ but the simultaneous 
^ presence of both makes them confounding factors which render some comparisons 
unclear,^ This situation cannot be remedied; it* can only be tolefa£e^d arid 



•JF. 
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considered in analysis. It will be, resolved next year by the fact that ACE 
students will graduate. The other issue related to student populations 
does have relevance to planning. An advance has been made this year by the 
very existance of any control groups, and this advance should not be under- 
estimated, but "pure" controls should be available next year to enhance the 
power^^^^tialytic conclusions. Kandon assignment to experiemental and 
cojcvttol groups is what is meant by the term "^pure". In order for this to 
be ng ssible recruitment must begin early, and the evaluation needs must be 
incorporated into the recruitment plan. 

Recommendation 3 . Evelry effort should be made to enable early re-- 
cruiting of students, and selection should be made* by randanr jtS^Tignment of 
qualified candidates tp experimental and control groups. ^' 
Instruments ^ _ 

Instrumentation often provades a rich source of difficulties for ex-* 
perimental projects. Given the options of standardized instruments or project 
, developed measures, most evaluators sele.ct a combination of the two giving 
recognition to the weaknesses of each and finding safety in numbers. The 
double J)ind of standardised instrument not measuring program objectives and 
program developed measures lacking statistical backup dn^SMneraliz^^lity " 
is seldom resolved. Few projects last long enough or have sufficient''' resources 
to produce technically sound measures which also allow valid compar^ori^ with 



non-proj-ect sufed'ects. The Career Education Programs are subject to/ these 

■ - 1 . \ 

difficulties. Significant advances in measurement Tiave been made over th^ 
last year with the active 5u|,{)iJQrt of NIE. One project is carefully investigat 
ing one standardized instrument which is in widespread use. Eahc laboratory 
has put resources in±o instrument development. The^^pic^^Jfe^s in concert have 




begun the development ^r^'>s^rnm23a^mea^ure But th.is represents only a beginning 
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and resources for such research have been limited, A more, concentrated and 
coordinated attack on the problem needs to be undertaken if it is to be solved, 
* The sufficiency, appropriatenss and implications of each, measure need to be 
investigated and explicated,. Some facets of the problem may be approached by 
NIE lietennining priorities and .criteria; given limited resources others may 
require a coordinated effort on the part of all participating projects. The 
effort would have significance for the presently imp lemen ted program as well 
as future measurement in~^ Career' Education, 

udat^on 4, The status of existing instrumentation should be assessed. 




The characteristics of standar^izg d i nstrumentation as they relate to program 

objectives and the valid determination of comparative program effects should 
be investigated, A coordinated effort to maximize the technical soundness and 

generalizeability of project-developed instrumentation should be undertaken, ' 

.Attitudinal Differences by Grade , * * 

Results from the. Student Opinion Survey indicate<i with consistency that 

twelfth graders' opinions of the_pr^gram were less favorable than tenth and 



eleventh graders'. The absense of comparative logitudinal data makes it 
impossible to co^nclude that second year students decline in regard for the 
program, but a .problem is definitely in evidence, . The consistent attitude 
differences and lower ^grades and attendances in some ar§as for twelfth graders 
suggests the need for some directed action. 

Recommendation 5 , Project staff should make a special effort through 

' . 

counseling, instructional and personal interaction activities to determine the 
nature and extent of dissatisfaction among twelfth graders. If the phenomenon 

can be casually, defined a concerted effort should be made to correct it. 
Recommendation 6 , Since it is possible that the observed phenomenon 
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"--4s_an^e3^^ effect, e.g., Hawthorne depreciation or cummulative measure- 

ment fatigue, it should be carefully observed next year, when comparable longitu- 
dinal data may be collected. 
Utilization of Evaluation Findings 

Although mos-t of the scheduled evaluation reports have yet to be produced 

^ this yedr,two associated concerns are briefly noted in. order to underline ,their 
importance* Last year^ evaluation reports were technically strong but weak in 
ready applicability to program planning an4 development; they required more 
transition to user needs. Also, a related "^oncern was determining the extent • 
to which evaluation reports were actually used by project participants. Both 
of these concerns have be^n ^tablished as pr^Lorities for this year, but pro- 
gress in meeting them has not been satisfactory. The extent of the task load 

has tended to deminish the time which can be de-^bted to results interpretation 

V * 
for different" audiences. A plan for assessing tH|e evaluation, utilization ha^^ 

yet to be developed. The importance of accomplis^ng these ^priorities should 

again be emphasized. 



Recommendation 7. The evaluation task load sh|uld be reexai^ined and 
prioritized to definitely permit the production of u&ir-refererfced reports and 
assessment of Evaluation utilization. The possibility Vf excizing some facets 
of the evaluation in-order to*enhance ^^hfesepriority 'ar^a^ should be considered 
particularly for next year. 
Basic Skills Development 

■ Comprehensive Tests of Basic Skills (CTBS). \ pretest -^^posttest Results 
were available only for the twelfth grade experimental students for the purposes 
of the present yeport. Analysis of the change scores indicated significant gains 
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over the 'five months of instruction covered. The observed better attendance 
and attitude of the tenth and eleventh graders would suggest that their gains 
will be even larger. Arithmetic skills, especially Arithmetic Computation 
and, Arithmetic Concepts, showed generally dimnished gains. Students who had 
noirbeen participating in the Individualized Learning Center activities ex- 
.hibited an overall loss. The lack of comparative data or tenth and eleventh 
graoe data hinder ' interpretation of these results. But, even in their pre- 
liminary form, they suf:gest soine action. 

Recommendation 8. Possible program deficiencies* in the areas of Teast 
evident basic skills student gains should be investigated. The program 
material or its utilization may be manipulated to better serve apparent student 
needs. • . > 

Recommendation 9 . la view of the fact that students not in the Indivi- 
dualized Learning Center seem to regress (some of it is artifactual)^^aon^ - - 
assignment ^_tlUs_ajC±j^^ very carefully. The alternatives to 

Center assignment ^should also be we U considered. 
Recruitmeat and Selection of Employers 

The current cluster system of 16 clusters and 48 employers appears to be 
sufficient to provide Career Exploration and Specialization activities for the 
current enrollment "of 134 students. However, the anticipated enrollment for 
next year is 300 students and the current cluster and employer system does not 
appear capable of previewing adequate Exploration anJ Specialization experiences 
for a student body of -that size; 

Recommendation 10 . Continuing efforts should b.e made to identify ex- 
perience clusters relevant to the Career Education Program and to recruit and 
select employers to both maintain and extend the cluster system. 
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Recruitment arid Selection of Students 

It is planned that the student enrollment in the Career Education Pro- 
gram will be 300 next year* Of these, 100 are expected to be 'students who are 
participating in the program this year. An additional 200 students must be 
enrolled to bring the total .enrollment to the desired leve], assuming that 
300 is a desired yiar-end enrollment. Past experience has indicated that twice 
the number desired must be accepted since only half the accepted students have 
enrolled. Attrition of students in the past has been at about a 30 perc^t 
rate over a year. Since the desired enrollment figure is a year-end figure, 
additional students should be accepted to account for expected attrition. Thej 
formula which has been derived is 2 x 1.3 x desired new enrollment of 200 
for both rate of enrollment and attrition. » 

A control group of 100 students is desired for purposes of making com-r 
parisons. Previous experience is that a third of control group students, have 
not been available for posttesting. To ensure a year end control group of a 
given size the following formula has been derived: 1.5x size of de^Sired con- 
trol group. 

-* Recommendation 11. 
.f^^ . 

A total of 670 students should be identified who could be accepted 

for tl\e Career Education Program; these 670 would meet the Career Education 

'Program's needs for enrollment and control group students. The figure of 670 

students was derived in the following manner: 

520 = 2 X 1.3 X desired new enrollment of 200 

-f 150 - 1.5 desired control groups of 100 

670 = the 'number of students who must be identified as being accep^table 

» 

for the program. 
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- ^ A student enrollment ' is desired which is repre^sentative of the racial 
composition of Philadelphia students. The racial composition of students 
recruited this year was 78 percent Black and 22 percent White. The racial 
composition of students recruited the previous year was 61 percent' Black and 
38 percent White. If this trend continues, the Career Education Program will 
have served only one racial subgroup of .the total population, 

a 

It is anticipated that selection of students for the Career Education 
Program next year will be a random seleQtion from a pool of stu^dents who meet 
entrance requirements of the program. 

Recommendation 12. Since the selection of students for the Career 
Education Program is to be on a random basis next year, efforts to obtain a 
representative racial composition will have to be focussed in £he area of 
recruitment. Efforts to recruit students should be directed toward all racial 
subgroups of the population, ^ ^ , 
Instructional Services 

A goal of the Career Education Program is the maximal utilizat:ypn by 
students of all components of the program. This year both attendance and 
achievement in the Basic Skills Unit were less than optimal. Attendance by 
10th and 11th grade and 12th grade students was slightly above 70 percent. 
Achievement, as indicated by rate of successful completion of credits for 

which students enrolled, was slightly above 70 percent for 12th grade students 
only, . 

Another goal of the Career Education Program is the participation by 
students in the specialization aspects 6f the Career Development and Career 
Guidance Units, This participation has been at a less than desired level so 
far this year. 
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Recommendation 13.- -^. 

Efforts should be made to increase the motivation of .students to both 
attend and achieve in Basic Skills Activities. Efforts to' increase motiva- 
tion have relied pn intrinsic factors in the past; an investigation of the 
pos'sible application of a combination of intrinsic and extrinsic motivational 
factors to the Basic Skills Unit should be conducted. ^ . 

Recommendation 14. ^"--^ ^ 

, Participation in the specialization aspects of the Career Dfevelopmeat 
4 and Career Guidance Programs should be strongly encouraged by. ^the Counselor- 
Coor<|<r?rators. Students should be made more aware of the personal benefits 
that can accrue to them by such, participation. 
Support Systems; Staff and 'Facilities ' • .^^^ 

An area of difficulty in staff interrelationships was a lack of definition 
in staff roles responsible for development ^nd implementation in the 
Career Guidance and Career Development Units* Complexities in the relationship 
^with the Greater Philadelphia Chamber of 'Commerce was seen as the source of 
this difficulty; questions of responsibility arose between the team leaders 
of the Career Development Unit, the Jateer Guidance Unit, and the Chamber's J 
Project Director., " " * '« ' v. 

Recommendation 15 . . ^ 

A more concisely defined relationship should be developed with the ^ 

Greater Philadelphia Chamber of CommeTc^^or ameliorate^any confusions re- ' 

' •* , ■^^ ^ 

garding respc risibility or accountability for the development and implementa- 
tion of Career Development and Career Guidance activites. The roles of the 
— lieam leaders qf the Career Development and Career Guidance Units and the 

Project Director of the Chamber should be defined in terms of explicit fqnction 
and related responsibilities. 
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Supplementary Activities 

> The Supplementary Activities are scheduled to be phased-out of the Career 
Education* Program wh^ 12tli grade students graduate this year. However, 
students seem*^ be' highly motivated to participate in this type .of program; 
attendance (90.7%) and^^s (B Average) are especially high for this program. 
Recommendation 16 . 

Means of incdrporating aspects of the Supplementary Activities into the 



other units of the: Career Educa 



tioh>^ 



rogram should be investigated. This 



recommendation might relate to means of increasing student motivation in 
the Basic Skills Unit (Recommendation 13). ^ 




